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Foreword 


In  this  volume  we  present  the  major,  policy-oriented  report  of  OTA's  assessment, 
The  Implications  of  Cost-Effectiveness  Analysis  of  Medical  Technology.  Begun  in  Oc- 
tober 1978,  the  assessment  analyzes  the  feasibility,  implications,  and  usefulness  of 
cost-effectiveness  and  cost-benefit  analysis  in  health  care  decisionmaking. 

In  addition  to  this  main  report,  there  will  be  five  background  papers:  1)  one  cov- 
ering methodology  and  literature  issues,  2)  a  psychotherapy  case  study,  3)  a  diagnostic 
X-ray  case  study,  4)  17  other  case  studies  of  individual  medical  technologies,  and  5)  a 
review  of  international  experience  in  managing  medical  technology.  Another  related 
study  was  published  in  September  of  1979:  A  Review  of  Selected  Federal  Vaccine  and 
Immunization  Policies. 

The  main  report,  The  Implications  of  Cost-Effectiveness  Analysis  of  Medical 
Technology,  was  prepared  by  OTA  staff.  In  preparing  this  report,  OTA  consulted 
with  members  of  the  advisory  panel  for  the  assessment,  with  the  authors  of  the  case 
studies  prepared  for  the  assessment,  and  with  numerous  other  experts  in  economics, 
medicine,  ethics,  and  health  policy. 

Drafts  of  the  final  report  were  reviewed  by  the  advisory  panel  chaired  by  Dr.  John 
Hogness,  by  the  Health  Program  Advisory  Committee  chaired  by  Dr.  Frederick  Rob- 
bins,  and  by  approximately  75  other  individuals  and  groups  representing  a  wide  range 
of  disciplines  and  perspectives  (see  Acknowledgments  in  app.  H).  We  are  grateful  for 
their  assistance. 
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Summary  and  Policy  Options 


Nature  never  gives  anything  to  anyone;  everything  is  sold.  It 
is  only  in  the  abstractions  of  ideals  that  choice  comes  without 
consequences. 

Ralph  Waldo  Emerson 
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1. 

Summary  and  Policy  Options 


The  rapid  and  continuing  growth  of  expendi- 
tures is  a  central  issue  in  many  policy  decisions 
concerning  the  medical  care  system  of  the 
United  States.  Policymakers,  health  profession- 
als, and  consumers  are  seeking  ways  to  control 
this  growth  while  simultaneously  improving  the 
quality  of  health  care.  Increasingly,  the  use  of 
cost-effectiveness  analysis/cost-benefit  analysis 
(CEA/CBA)  is  being  advocated  as  a  possible 
means  of  making  the  medical  care  system  more 
efficient.  In  particular,  this  technique  is  sug- 
gested for  use  in  health  care  programs — for  ex- 
ample, by  the  medicare  program  in  its  reim- 
bursement coverage  decisions.  Nevertheless,  a 
great  deal  of  confusion  and  disagreement  sur- 
rounds the  implications  and  feasibility  of  apply- 
ing CEA/CBA  in  health  care.  To  aid  in  their  de- 
cisions concerning  the  possible  use  of  CEA/ 
CBA  in  Federal  health  programs,  the  Senate 
Committees  on  Labor  and  Human  Resources 
and  on  Finance  asked  OTA  to  explore  the  appli- 
cability of  CEA/CBA  to  medical  technology. 

In  the  assessment,  three  major  issues  are  ex- 
amined: 1)  the  general  value  of  CEA/CBA  in  de- 
cisionmaking about  the  use  of  medical  technol- 
ogy; 2)  the  methodological  strengths  and  short- 
comings of  the  technique;  and  3)  the  potential 
for  initiating  or  expanding  the  use  of  CEA/ 
CBA  in  six  health  care  programs — reimburse- 
ment coverage,  the  Professional  Standards  Re- 
view Organizations  (PSROs),  health  planning, 
market  approval  for  drugs  and  medical  devices, 


R&D  activities,  and  health  maintenance  organi- 
zations (HMOs) — and,  most  importantly,  the 
implications  of  any  expanded  use. 

The  prime  focus  of  the  assessment  is  on  the 
application  of  CEA/CBA  to  medical  technol- 
ogy, i.e.,  the  drugs,  devices,  medical  and  surgi- 
cal procedures  used  in  medical  care,  and  the  or- 
ganizational and  support  systems  within  which 
such  care  is  provided.  Except  in  a  background 
paper  on  psychotherapy,  the  report  does  not  ad- 
dress psychosocial  medicine.  Other  factors  in- 
fluencing health,  such  as  the  environment,  are 
not  directly  covered  either.  The  findings  of  this 
assessment,  though,  might  very  well  apply  to 
health  care  resource  decisionmaking  in  general, 
and  with  modification,  to  other  policy  areas 
such  as  education,  the  environment,  and  occu- 
pational safety  and  health. 

This  OTA  assessment  finds  that  CEA/CBA 
cannot  serve  as  the  sole  or  primary  determinant 
of  a  health  care  decision.  Decisionmaking  could 
be  improved,  however,  by  the  process  of  identi- 
fying and  considering  all  the  relevant  costs  and 
benefits  of  a  decision.  At  present,  using  the  ap- 
proach or  process  of  CEA/CBA  in  decisionmak- 
ing may  be  more  helpful  than  the  rigid  and  for- 
mal application  of  CEA/CBA  study  results  in 
health  care  program  decisions.  It  is  unrealistic, 
moreover,  to  expect  that  CEA/CBA,  in  itself, 
would  be  an  effective  tool  for  reducing  or  con- 
trolling overall  expenditures  for  medical  care. 


EVALUATING  COSTS  AND  BENEFITS 


All  decisions  have  consequences.  Usually, 
however,  in  most  decisionmaking  processes 
only  a  fraction  of  all  potential  consequences  is 
taken  into  account.  The  inherent  complexities 
and  uncertainties  associated  with  many  deci- 
sions make  it  extremely  difficult  to  identify  and 
weigh  all  possible  consequences.  In  general, 
however,  the  quality  or  validity  of  decisions  can 


be  increased  by  analysis  that  forces  a  structuring 
of  the  decision  process — that  provides  a  consist- 
ent framework  for  identifying  and  considering 
as  many  of  the  relevant  costs  and  benefits  as  is 
feasible. 

The  public,  or  governmental,  sector  is  called 
upon  to  make  certain  decisions  that  are  imprac- 


3 


4  •  The  Implications  of  Cost-Effectiveness  Analysis  of  Medical  Technology 


tical  for  the  private  sector  to  make.  Examples  of 
these  are  decisions  concerning  national  defense 
or  air  pollution  control,  neither  of  which  is 
amenable  to  being  traded  in  the  marketplace. 
Other  decisions  are  made  by  the  public  sector 
for  social  reasons  such  as  assuring  equitable  dis- 
tribution of  what  may  be  considered  essential 
goods  (e.g.,  health  care  for  the  elderly).  Because 
conventional  private  sector  techniques,  such  as 
capital  budgeting  and  return-on-investment 
analysis,  are  insufficient  for  these  decisions, 
techniques  such  as  CBA  and  CEA  have  been  de- 
veloped. In  the  medical  care  area,  CEA/CBA  is 
designed  to  integrate  the  economic  aspects  of  a 
decision  with  the  health  aspects  of  that  decision. 
Consequently,  it  should  not  be  considered  sim- 
ply an  economic  tool. 

There  are  two  basic  types  of  health  care  re- 
source allocation  decisions  which  in  theory 
could  benefit  from  a  CEA/CBA  process.  The 
first  are  decisions  made  within  a  fixed  or  pro- 
spectively set  budget,  such  as  those  made  by 
HMOs.  The  second  are  decisions  made  in  the 


absence  of  a  direct  budget  constraint,  such  as 
those  made  for  medicare  reimbursement  or  in 
health  planning. 

In  the  former — allocation  decisions  made 
within  a  budget — tradeoffs  must  be  made,  since 
not  all  projects  can  be  funded.  The  projects  that 
promise  to  deliver  more  benefits  for  the  cost 
should  be  more  attractive  than  those  projects 
expected  to  deliver  fewer  benefits.  In  these  deci- 
sions, the  function  of  CEA/CBA  would  be  to  il- 
luminate the  decision  process  and  to  require  that 
implicit  judgments  be  made  more  explicitly, 
thus  forcing  external  examination  of  the  as- 
sumptions and  values  placed  on  decision  vari- 
ables. 

In  nonconstrained  decisions,  direct  tradeoffs 
between  competing  projects  often  are  unneces- 
sary. Consequently,  a  function  of  CEA/CBA  in 
these  decisions  would  be  to  force  consideration 
of  economic  factors.  In  health  planning  deci- 
sions, for  example,  planners  would  be  asked  to 
consider  not  only  whether  a  service  is  needed 
but  also  whether  it  is  worth  the  cost. 


COST-BENEFIT  ANALYSIS  AND  COST-EFFECTIVENESS  ANALYSIS 


The  terms  CBA  and  CEA  refer  to  formal  ana- 
lytical techniques  for  comparing  the  positive 
and  negative  consequences  of  alternative  ways 
to  allocate  resources.  In  practice,  the  compar- 
ison of  costs  and  benefits  is  accomplished 
through  various  analytical  approaches,  which 
comprise  a  spectrum  ranging  from  sophisticated 
computer-based  analysis  using  large  amounts  of 
epidemiological  and  other  data  to  partially  in- 
tuitive, best-guess  estimates  of  costs  and  bene- 
fits. Some  analyses  may  take  into  account  the 
results  of  clinical  trials  of  a  technology  and 
model  the  technology's  effect  on  health  out- 
comes. Others  may  assume  that  the  alternative 
technologies  under  study  have  equal  effective- 
ness and  concentrate  on  the  difference  in  costs 
involved. 

Thus,  there  is  a  continuum  of  analyses  that 
examine  costs  and  benefits.  At  one  end  of  the 
continuum  are  what  will  be  referred  to  as  "net 
cost"  studies.  In  these  studies  the  emphasis  is  on 


costs,  and  net  cost  studies  in  the  past  have  often 
assumed  benefits  or  efficacy  to  be  equal.  At  the 
other  end  of  the  continuum  are  analyses  that  at- 
tempt to  relate  the  use  of  the  technologies  under 
study  to  specific  health-related  outcomes  and  to 
compare  the  costs  of  the  technologies  to  the  dif- 
ferences in  health  benefits.  CBA  and  CEA  com- 
prise the  entire  set  of  analytical  techniques — 
differentiated  by  the  specifics  of  what  costs  and 
benefits  are  considered  and  how  they  are  ana- 
lyzed— on  this  continuum. 

The  principal  distinctions  between  CEA  and 
CBA  lie  in  the  valuation  of  the  desirable  conse- 
quences of  a  decision,  in  the  implications  of  the 
different  methods  of  that  valuation,  and  usually 
in  the  scope  of  the  analysis.  In  CBA  all  costs  and 
all  benefits  are  valued  in  monetary  terms.  Thus, 
conceptually,  CBA  can  be  used  to  evaluate  the 
"worth"  of  a  project  and  would  allow  compar- 
ison of  projects  of  different  types  (such  as  dams 
and  hospitals).  In  CEA,  the  health-related  ef- 
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fects  of  programs  or  technologies  are  not  valued 
in  monetary  terms  but  rather  are  measured  in 
some  other  unit  (such  as  years  of  life  gained).  A 
CEA,  therefore,  does  not  result  in  a  net  mone- 
tary value  for  a  project.  Instead,  it  produces  a 
measure  of  the  cost  involved  in  attaining  some 
desirable  health-related  effect.  Conceptually, 


CEA  permits  direct  comparison  of  only  those 
programs  or  technologies  that  share  similar  ob- 
jectives. This  OTA  assessment  uses  the  term 
CEA/CBA  to  refer  to  both  of  these  techniques 
because  the  findings  below  apply  generally  to 
both. 


FINDINGS 

General 

Most  of  the  specific  findings  presented  below 
relate  to  the  two  major,  general  findings  of  the 
OTA  assessment: 

•  Performing  an  analysis  of  costs  and  bene- 
fits can  be  very  helpful  to  decisionmakers 
because  the  process  of  analysis  gives  struc- 
ture to  the  problem,  allows  an  open  consid- 
eration of  all  relevant  effects  of  a  decision, 
and  forces  the  explicit  treatment  of  key 
assumptions. 

•  CEA/CBA  exhibits  too  many  methodologi- 
cal and  other  limitations,  however,  to  justi- 
fy relying  solely  on  the  results  of  formal 
CEA/CBA  studies  in  making  a  decision. 
Thus,  although  CEA/CBA  is  useful  for  as- 
sisting in  many  decisions,  it  should  not  be 
the  sole  or  prime  determinant  of  a  decision. 

CEA/CBA  is  viewed  by  different  parties  as 
ranging  in  usefulness  from  obfuscating  the  perti- 
nent issues  in  a  decision  process  at  one  extreme 
to  illuminating  and  synthesizing  these  issues  so 
well  that  the  technique  can  be  used  to  make  de- 
cisions at  the  other  extreme.  There  is,  however, 
a  middle  position  which  maintains  that  the  tech- 
nique could  be  helpful  in  structuring  informa- 
tion and  that  this  information  should  be  only 
one  of  several  components  of  a  decision  process. 
The  OTA  case  study  on  the  artificial  heart,  for 
example,  lays  out  many  of  the  factors  to  be  con- 
sidered in  decisions  regarding  continued  funding 
of  R&D  of  this  technology.  But  those  decisions 
will  also  be  dependent  on  a  number  of  other  po- 
litical and  social  factors  that  are  difficult  to  ana- 
lyze systematically. 

Both  extreme  positions  mentioned  above  per- 
tain to  the  use  of  CEA/CBA  as  a  formal,  struc- 


tured analysis  which  is  oriented  toward  a 
bottom-line  answer,  such  as  a  cost-benefit  ratio. 
Such  a  bottom  line,  however,  often  hides  many 
important  value  judgments,  thus  providing  a 
seemingly  unambiguous  answer  that  may  rest 
on  ambiguous  data  or  assumptions. 

Advocates  of  the  middle  position  propose 
that  CEA/CBA  be  used  within  the  context  of  ac- 
cepted principles  of  analysis  in  order  to  illum- 
inate the  costs  and  the  benefits  of  a  decision  but 
not  necessarily  to  aggregate  and  weigh  them. 

The  findings  of  this  and  any  other  assess- 
ments of  the  past  usefulness  or  current  potential 
of  CEA/CBA  in  health  care  decisionmaking 
should  be  kept  in  perspective:  Because  there  has 
been  little  experience  with  the  use  of  CEA/CBA, 
these  findings  are  based  on  very  little  definite  in- 
formation. Therefore,  any  of  the  findings  or 
projections  of  the  usefulness  or  implications  of 
CEA/CBA  in  this  report  could  be  proven  wrong 
as  more  experience  accumulates.  In  fact,  one  of 
the  priorities  for  future  examinations  of  the  role 
of  CEA/CBA  in  health  care  should  perhaps  be 
small,  controlled  experiments,  or  demonstra- 
tions, of  its  potential  use  (see  Option  10  below). 

Methodology 

There  is  no  set  combination  of  specific  analyt- 
ical elements  that  form  a  standard  CEA  or  CBA 
methodology.  A  standard  or  rigid  methodolo- 
gy, however,  is  to  be  neither  expected  nor  de- 
sired. OTA  found  a  wide  variation  in  the  forms 
of  CEA/CBA  analyses  actually  conducted  to 
date.  Most  of  the  analyses  reviewed  seem  to  be 
academic  exercises,  infrequently  connected  to 
the  policy  process.  In  addition,  the  literature  on 
CEA/CBA  indicates  that  a  great  many  of  the 
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analyses  conducted  tend  to  be  on  the  "net  cost" 
end  of  the  CEA/CBA  continuum,  that  is,  anal- 
yses in  which  health  outcomes  were  not  taken 
into  explicit  account.  In  part,  this  may  be  a  re- 
flection of  the  paucity  of  data  on  the  efficacy, 
safety,  and  appropriate  use  of  medical  tech- 
nology. 

OTA's  assessment  does  find,  however,  gen- 
eral agreement  on  a  set  of  10  principles  of 
analysis  that  could  be  used  to  guide  the  conduct, 
evaluation,  or  use  of  CEA/CBA  studies.  OTA 
believes  that  use  of  generally  accepted  principles 
is  important,  largely  because  of  the  basic  meth- 
odological limitations  of  CEA/CBA.  For  exam- 
ple, although  the  results  of  a  CEA/CBA  are  of- 
ten dependent  on  the  discount  rate  chosen,  there 
is  no  general  agreement  on  what  discount  rate 
should  be  used  under  what  circumstances.  Fur- 
thermore, as  primarily  an  efficiency-oriented 
technique,  CEA/CBA  is  weak  in  the  areas  of 
equity  and  other  ethical  considerations.  Finally, 
CEA/CBA  must  often  address  issues  of  great 
uncertainty,  such  as  disease  progression,  patient 
compliance  rates,  differing  responses  to  tech- 
nology by  differing  population  groups,  and  so 
on.  These  and  other  limitations  must  be  kept  in 
mind  and  must  to  the  maximum  extent  feasible 
be  compensated  for  by  techniques  such  as  sen- 
sitivity analysis  (see  #8  below).  The  10  basic 
principles  of  analysis  are: 

1.  Define  Problem. — The  problem  should 
be  clearly  and  explicitly  defined  and  the 
relationship  to  health  outcome  or  status 
should  be  stated. 

2.  State  Objectives. — The  objectives  of  the 
technology  being  assessed  should  be  ex- 
plicitly stated,  and  the  analysis  should 
address  the  degree  to  which  the  objec- 
tives are  (expected  to  be)  met. 

3.  Identify  Alternatives. — Alternative 
means  (technologies)  to  accomplish  the 
objectives  should  be  identified  and  sub- 
jected to  analysis.  When  slightly  dif- 
ferent outcomes  are  involved,  the  effect 
this  difference  will  have  on  the  analysis 
should  be  examined. 

4.  Analyze  Benefits/Effects. — All  foresee- 
able benefits/effects  (positive  and  nega- 
tive outcomes)  should  be  identified,  and 


when  possible,  should  be  measured. 
Also,  when  possible,  and  if  agreement 
can  be  reached,  it  may  be  helpful  to  val- 
ue all  benefits  in  common  terms  in  order 
to  make  comparisons  easier. 

5.  Analyze  Costs. — All  expected  costs 
should  be  identified,  and  when  possible, 
should  be  measured  and  valued  in  dol- 
lars. 

6.  Differentiate  Perspective  of  Analysis. — 

When  private  or  program  benefits  and 
costs  differ  from  social  benefits  and  costs 
(and  if  a  private  or  program  perspective 
is  appropriate  for  the  analysis),  the  dif- 
ferences should  be  identified. 

7.  Perform  Discounting. — All  future  costs 
and  benefits  should  be  discounted  to 
their  present  value. 

8.  Analyze  Uncertainties. — Sensitivity 
analysis  should  be  conducted.  Key  vari- 
ables should  be  analyzed  to  determine 
the  importance  of  their  uncertainty  to  the 
results  of  the  analysis.  A  range  of  possi- 
ble values  for  each  variable  should  be  ex- 
amined for  effects  on  results. 

9.  Address  Ethical  Issues. — Ethical  issues 
should  be  identified,  discussed,  and 
placed  in  appropriate  perspective  relative 
to  the  rest  of  the  analysis  and  the  objec- 
tives of  the  technology. 

10.  Discuss  Results. — The  results  of  the  anal- 
ysis should  be  discussed  in  terms  of  va- 
lidity, sensitivity  to  changes  in  assump- 
tions, and  implications  for  policy  or  deci- 
sionmaking. 

In  addition  to  conforming  to  these  10  prin- 
ciples, all  quantitative  analyses  should  speci- 
fy data  sources  and  be  written  as  clearly  and 
nontechnically  as  possible.  They  should  also  be 
subjected  to  review — including  public  scruti- 
ny when  appropriate — especially  regarding  as- 
sumptions upon  which  their  outcomes  may  rest. 

OTA  found  that  many  of  the  methodological 
limitations  of  CEA/CBA  are  often  hidden  by 
the  practice  of  deriving  a  cost-benefit  or  cost- 
effectiveness  ratio — that  is,  a  numerical  bottom- 
line.  Therefore,  the  possibility  of  not  aggregat- 
ing the  often  complex  sets  of  calculations  should 
be  investigated.  Instead  of  aggregating,  analyses 
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might  be  done  by  explicitly  listing  or  ARRAY- 
ING all  the  elements  which  are  included  in  or 
would  be  affected  by  a  decision.  Where  costs 
and  effects  could  be  quantified,  that  would  be 
done;  where  they  could  be  combined,  that 
would  be  done.  But  no  effort  would  be  made  to 
arrive  at  a  single  combined  value  when  one  or 
more  important  nonquantifiable  variables  that 
would  have  to  be  left  out  or  relegated  to  a  foot- 
note. Such  an  array  method  of  analysis  would 
mean  that  decisionmakers  would  have  a  greater 
number  of  elements  to  consider,  but  it  would 
also  mean  that  intangible  or  nonquantifiable 
factors  would  be  more  explicit  and,  thus,  more 
likely  to  be  taken  into  consideration. 

The  findings  of  this  assessment,  especially 
methodological  ones,  focus  primarily  on  the 
practical  implications  of  CEA/CBA  for  health 
policy.  The  report  is  not  written  for  the  aca- 
demic research  community.  The  general  prin- 
ciples above  apply  to  analysis  for  policy  use.  In 
no  way,  however,  should  the  findings  be  taken 
to  mean  that  complex,  sophisticated  analysis  is 
always  unnecessary  or  superfluous.  Advance- 
ment of  the  state-of-the-art  of  CEA/CBA  re- 
quires ongoing  and  sophisticated  research.  Cur- 
rent research  on  the  development  and  validation 
of  sets  of  indexes  of  the  health  status  of  a  pop- 
ulation, for  example,  appears  very  valuable. 
Use  of  such  complex  aspects  of  CEA/CBA, 
however,  may  require  greater  technical  capa- 
bilities than  most  health  programs  currently 
possess.  Addressing  a  more  limited  approach  to 
analysis,  then,  seems  appropriate  for  the  goal  of 
this  assessment.  But  it  does  not  diminish  the 
need  for  more  complex  approaches  in  research 
or  other  specialized  circumstances. 

Applying  CEA/CBA  in  Health  Care 

As  stated  above,  CEA/CBA  can  be  more  val- 
uable to  health  care  decisionmaking  when  it 
serves  as  a  problem  structuring  process  than 
when  it  becomes  the  primary  factor,  with  nu- 
merical results,  of  a  decision.  Furthermore, 
CEA/CBA  potentially  can  be  more  valuable  for 
decisionmaking  under  a  constrained  budget, 
when  tradeoffs  have  to  be  made  directly,  than 
when  constraints  are  nonexistent  or  very  indi- 


rect. In  neither  case,  however,  would  CEA/ 
CBA  necessarily  function  as  an  effective  cost- 
constraining  mechanism  or  tool.  Under  the 
budget  system,  the  budget  itself  would  be  the 
constraining  mechanism.  Under  the  noncon- 
strained  system,  since  no  direct  tradeoffs  are  re- 
quired, no  direct  limit  on  expenditures  is  set. 
CEA/CBA  might,  however,  change  the  mix  of 
expenditures.  Especially  under  a  budget  system, 
technologies  might  be  substituted  for  one 
another  as  a  result  of  analysis. 

The  context  within  which  a  decision  will  take 
place  must  be  specified  before  any  judgment  of 
the  usefulness  or  applicability  of  CEA/CBA  can 
be  made.  For  example,  does  the  decision  relate 
to  a  technology  at  an  early  stage  in  its  lifecycle 
such  as  bone  marrow  transplants?  Or  does  it 
concern  an  established  technology,  such  as  cer- 
vical cancer  screening?  Is  the  technology  in 
question  a  diagnostic  technology,  such  as  the 
CT  scanner,  or  a  therapeutic  one,  such  as  renal 
dialysis?  The  possibility  of  affecting  the  course 
of  a  technology's  diffusion  and  use  might  be 
greater  in  early  stages  of  its  development,  but 
the  uncertainties  about  its  health  effects  and  its 
costs  will  generally  be  greater.  Thus,  it  may  be 
possible  to  do  a  more  valid  or  certain  CEA/ 
CBA  later  in  the  technology's  lifecycle,  but  the 
information  gained  may  be  less  valuable  for 
public  policy.  The  tradeoff  required  will  vary 
depending  on  the  specifics  of  the  technology  and 
the  policy  decision  to  be  made.  Similarly,  diag- 
nostic technologies  are  often  more  difficult  to 
study  than  other  types  because  of  the  uncertain- 
ties involved  in  linking  their  use  to  health  out- 
comes. Thus,  studies  of  diagnostic  technologies 
often  tend  toward  the  "net  cost"  end  of  the 
CEA/CBA  spectrum,  where  the  measures  of 
outcome  or  benefit  may  be  numbers  of  tests  per- 
formed or  levels  of  diagnostic  accuracy. 

Applying  CEA/CBA  in  Specific  Health 
Care  Programs 

OTA  examined  the  current  and  potential  use 
of  CEA/CBA  or  related  techniques  in  six  health 
care  programs  (see  chs.  5  through  10).  Although 
informal  and  often  implicit  analysis  of  costs  and 
benefits  frequently  takes  place  in  health  policy 
formulation,  OTA  found  very  little  formal  use 
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of  CEA/CBA  in  the  programs  studied.  In  sever- 
al of  the  areas,  cost  itself  has  played  little  or  no 
role  in  policy  decisions. 

Reimbursement  programs  such  as  medicare 
and  Blue  Cross/Blue  Shield,  when  deciding 
what  technologies  will  be  covered,  concentrate 
on  criteria  that  generally  do  not  explicitly  in- 
clude costs  (e.g.,  stage  of  development  of  the 
technology  and  acceptance  by  the  medical  com- 
munity). Under  medicare,  initial  responsibility 
for  identifying  questions  about  whether  a  tech- 
nology should  be  covered  lies  with  a  system  of 
local  contractors  who  administer  the  program. 
When  not  resolved  at  the  local  level,  a  question 
is  referred  to  the  Health  Care  Financing  Admin- 
istration (HCFA),  which  may  seek  a  recommen- 
dation from  the  Public  Health  Service  (PHS). 
PHS  has  traditionally  used  four  criteria  in  its 
recommendations:  efficacy,  safety,  stage  of  de- 
velopment, and  acceptance  by  the  medical  com- 
munity. Other  health  insurance  programs,  such 
as  Blue  Cross/Blue  Shield,  operate  similarly. 

The  possibility  of  expanding  coverage  criteria 
to  explicitly  include  costs  or  cost  effectiveness  is 
being  examined  by  HCFA  and  PHS.  The  first 
question  to  be  answered  is  whether  there  is  a 
legal  basis  for  any  such  inclusion.  Currently,  the 
language  of  the  Social  Security  Act  requires  the 
medicare  program  to  pay  for  services  that  are 
"reasonable  and  necessary."  There  is  no  defini- 
tive interpretation  of  whether  that  language 
means  that  the  relative  "cost  effectiveness"  of  a 
particular  technology  might  make  it  unreason- 
able or  unnecessary. 

Most  current  reimbursement  programs  are  ex- 
amples of  programs  without  direct  budget  con- 
straints. Each  reimbursement  coverage  decision 
does  not  involve  a  tradeoff.  Approval  of  one 
technology  does  not  mean  that  another  will  not 
be  covered.  In  a  very  real  sense,  the  existing 
reimbursement  system  is  an  open-ended  system 
of  financing  medical  care. 

The  PSRO  program  was  enacted  to  assure 
that  health  services  provided  under  medicare, 
medicaid,  and  certain  other  programs  are  medi- 
cally necessary,  meet  professionally  recognized 
standards  of  care,  and  are  provided  at  the  most 
economical  level  consistent  with  quality  care. 


CEA/CBA  approaches  have  theoretical  applica- 
bility in  three  areas:  1)  the  development  of 
standards  of  care  against  which  actual  practices 
are  judged,  2)  the  internal  management  of  in- 
dividual PSROs,  and  3)  evaluations  of  the  na- 
tional PSRO  effort.  Cost-effectiveness  criteria 
have  not  been  directly  incorporated  into  stand- 
ards of  care  except  in  a  few  instances  (e.g.,  res- 
piratory therapy),  but  it  appears  possible  for 
PSROs  to  do  so,  although  the  studies  would 
most  likely  have  to  be  conducted  elsewhere. 
PSROs  generally  do  not  have  the  analytical 
capabilities  for  CEA/CBA.  Net  cost  techniques 
have  been  used  to  evaluate  whether  the  savings 
achieved  outweigh  the  costs  of  administering 
the  review  activities  of  the  overall  PSRO  pro- 
gram. These  analyses,  which  often  reach  con- 
tradictory conclusions,  do  not,  however,  ex- 
amine costs  in  relation  to  changes  in  health  out- 
comes that  may  result  from  PSRO  reviews.  Nor 
do  they  address  the  fact  that  even  the  most  op- 
timistic reports  of  savings  represent  an  infinites- 
imal portion  of  total  medicare  and  medicaid  ex- 
penditures. 

Because  individual  PSROs  operate  under  a 
program  management  budget,  incentives  ex- 
ist for  them  to  use  cost-effectiveness-like  ap- 
proaches in  choosing  areas  in  which  to  concen- 
trate their  review  activities. 

In  contrast  to  PSROs,  in  the  area  of  health 
planning,  the  National  Health  Planning  and 
Resources  Development  Act,  with  its  amend- 
ments, explicitly  states  that  resources  are  to  be 
allocated  in  a  more  efficient  manner  and  that 
health  planners  should  weigh  both  costs  and 
benefits  in  their  decision  processes.  The  Health 
Resources  Administration  (HRA)  is  emphasiz- 
ing a  more  analytical  approach  to  health  plan- 
ning, especially  in  regard  to  capital  budgeting. 
OTA  found  that  State  health  planning  and  de- 
velopment agencies  (SHPDAs)  and  health  sys- 
tems agencies  (HSAs)  are  for  the  most  part  still 
primarily  health  "needs"  oriented.  An  OTA  sur- 
vey of  planning  agencies  indicated  that  few 
agencies,  even  those  in  the  vanguard  of  using 
economic  analysis  for  allocation  of  resources, 
are  going  beyond  the  practice  of  considering 
only  capital  costs.  There  is  a  slight  trend,  how- 
ever, for  the  most  analytically  advanced  staffs 
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to  consider  the  marginal  (or  incremental)  costs 
associated  with  changes  in  use  of  a  technology. 
The  analysis  that  took  place  around  the  CT 
scanner  is  a  good  example  of  that.  OTA  discov- 
ered no  HSAs  that  explicitly  balance  costs  with 
health  benefits  in,  for  example,  certificate-of- 
need  recommendations.  Thus,  although  there 
appear  to  be  no  legal  barriers  to  its  use,  CEA/ 
CBA  has  not  been  much  applied.  In  health  plan- 
ning, as  in  reimbursement,  there  is  no  direct 
budget  constraint,  i.e.,  the  area  served  by  an 
HSA  is  not  operating  with  a  fixed  or  predeter- 
mined amount  of  resources  to  be  spent  on  health 
care.  Few  pressures,  therefore,  act  on  planning 
agencies  to  force  consideration  of  how  to  get  the 
most  health  effect  for  the  fewest  dollars. 

Market  approval  for  drugs  and  medical  de- 
vices, by  the  Food  and  Drug  Administration 
(FDA),  is  an  area  where  Congress  has  specified 
the  decision  criteria,  which  explicitly  do  not  in- 
clude costs.  FDA  is  required  to  regulate  the  mar- 
ket introduction  of  drugs  and  devices  on  the 
basis  of  effectiveness  (efficacy)  and  safety.  FDA 
has  not  formally  used  cost-effectiveness  or  any 
other  economic  criteria  to  evaluate  drug  or 
device  applications.  The  primary  purpose  of 
FDA  is  to  protect  the  public  from  unsafe  and  in- 
efficacious products.  Although  the  Agency's 
processes  do  have  indirect  influence  on  the  way 
resources  are  allocated,  there  are  several  factors 
arguing  against  the  incorporation  of  explicit 
economic  criteria  in  its  decision  processes.  Per- 
haps the  most  important  of  these  is  that  by  in- 
corporating such  criteria,  FDA  might  be  extend- 
ing its  regulatory  functions  beyond  those  envi- 
sioned by  Congress.  Further,  the  administrative 
burden  and  demands  of  doing  so  would  be  enor- 
mous. 

The  Federal  health  R&D  effort  encompasses  a 
broad  spectrum  of  activities  and  involves  sever- 
al Federal  agencies.  At  one  end  of  the  spectrum 
is  biomedical  and  behavioral  research  which  is 
supported  by  the  National  Institutes  of  Health, 
and  at  the  other  end  is  health  services  research 
such  as  that  supported  by  the  National  Center 
for  Health  Services  Research,  the  National  Cen- 
ter for  Health  Statistics,  and  HFCA.  Lying 
somewhere  in  the  middle  of  the  spectrum  and  in- 


corporating both  end  points  is  research  sup- 
ported by  the  National  Center  for  Health  Care 
Technology.  The  Federal  agencies  involved  in 
health  R&D  rarely  use  explicit  CEA/CBA  con- 
siderations to  set  research  priorities,  to  allocate 
research  resources,  or  to  evaluate  the  results  of 
research.  The  uncertain  end  products  of  much 
research  makes  it  difficult  to  conduct  CEA/ 
CBA.  Technologies  being  assessed  or  to  be  de- 
veloped are  often  at  an  early  stage  of  the  life- 
cycle.  A  CEA/CBA-like  approach,  with  no  re- 
quirement for  aggregation  of  variables,  may  be 
more  applicable  for  R&D  than  is  rigid,  tradi- 
tional CEA/CBA.  The  more  basic  the  level  of 
research,  the  less  applicable  the  techniques  be- 
come, owing  to  the  increased  uncertainties. 
Ironically,  it  may  frequently  be  more  desirable 
to  assess  a  technology,  including  its  cost  impli- 
cations, earlier  in  its  development  rather  than 
later. 

HMOs  are  both  insurers  and  providers  of 
medical  care.  They  are  an  example  of  a  program 
operating  under  a  constrained  budget  and  have 
a  direct  economic  incentive  to  provide  "cost-ef- 
fective" care.  However,  a  preliminary  OTA  ex- 
amination of  HMO's  decisionmaking  criteria  in- 
dicates that,  at  least  in  part  because  HMOs  op- 
erate in  a  predominantly  fee-for-service  envi- 
ronment and  must  compete  for  enrollees,  these 
organizations  do  not  commonly  weigh  health 
benefit  against  cost  in  deciding  what  medical 
services  to  offer.  The  actual  analytical  approach 
used  by  HMOs  seems  to  be  related  to  "net  cost" 
techniques.  Although  HMOs  do  not  always 
consider  health  benefits,  their  use  of  a  "net  cost" 
approach  suggests  that  they  view  the  provision 
of  technology  in  terms  of  efficiency.  They  seek 
ways  to  provide  benefits  comparable  to  fee-for- 
service  medicine  at  the  lowest  cost  feasible.  The 
current  potential  for  use  of  CEA/CBA  by 
HMOs,  weighing  health  benefits  against  costs, 
does  not  appear  to  be  as  large  as  the  existence  of 
direct  budget  constraints  would  predict.  That 
situation  may  change,  however,  as  more  experi- 
ence is  gained  with  CEA/CBA  and  as  HMOs  in- 
creasingly encounter  competition  with  each 
other  in  addition  to  fee-for-service  health  care 
providers. 
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POLICY  OPTIONS 

Options  for  congressional  consideration  fall 
into  two  categories:  1)  those  that  relate  to  the 
current  possibilities  for  using  CEA/CBA  in  pol- 
icy formulation  and  decisionmaking  (Options  1 
through  6),  and  2)  those  that  relate  to  the  devel- 
opment of  CEA/CBA  techiques  in  themselves 
(Options  7  through  10). 

Options  Relating  to  Current 
Possibilities  for  Use  of  CEA/CBA 

Options  1  through  6  follow  a  four-stage  pro- 
gression of  confidence  in  the  current  applicabili- 
ty of  CEA/CBA  to  health  care  programs — from 
a  perception  that  its  use  should  be  discouraged 
(Option  1),  to  a  neutral,  status  quo  attitude 
(Option  2),  to  cautious  encouragement  (Option 
3),  to  a  positive  belief  that  CEA/CBA  can  be  ef- 
fectively applied  (Options  4  through  6).  Options 
1  through  3  present  general  approaches  that  are 
mutually  exclusive  for  any  given  program  area. 
Each  of  Options  4  through  6,  however,  relates 
to  a  specific  program  area. 

OPTION  1 

Discourage  or  prohibit  the  explicit  inclusion  of 
cost-effectiveness  criteria  in  specified  health  care 
resource  allocation  decisions. 

The  implicit  weighing  of  the  costs  and  ben- 
efits of  resource  allocation  decisions  cannot — 
and  should  not — be  legislated  out  of  the  policy 
process.  It  is  inherently  a  part  of  that  process. 
What  this  option  does,  therefore,  is  signal  that 
the  techniques  available  to  formalize  the  now 
informal  process  of  weighing  these  costs  and 
benefits  are  not  regarded  as  sufficiently  valid  to 
justify  their  use. 

For  those  types  of  decisions  where  costs  have 
not  previously  been  explicitly  considered  to  any 
significant  extent  (such  as  medicare  coverage  of 
specific  technologies),  this  option  would  reflect 
a  positive  decision  not  to  include  costs  explicit- 
ly. For  decisions  made  within  a  constrained 
budget,  where  the  costs  of  alternative  alloca- 
tions of  resources  are  automatically,  though 
often  implicitly,  factored  in,  this  option  would 
be  a  statement  that  existing  methods  of  consid- 
ering costs  are  either  adequate  or  would  not  be 


improved  upon  by  mandatory  use  of  formal 
analytical  techniques. 

Reflected  in  this  option  are  a  recognition  of 
both  the  inherent  methodological  limitations  of 
CEA/CBA  and  the  weaknesses  that  result  from 
the  fact  that  CEA/CBA  is  still  an  evolving  meth- 
odology; a  belief  that  other  factors  in  the  deci- 
sionmaking context  are  more  important  than 
CEA/CBA-derived  information;  and  a  desire  to 
avoid  the  possible  misuse  of  CEA/CBA.  These 
considerations,  along  with  considerations  of  the 
country's  limited  experience  with  making  health 
care  allocation  decisions  based  on  CEA/CBA, 
and  the  expense  associated  with  conducting 
large  numbers  of  CEA/CBAs  required  for  deci- 
sionmaking, all  support  this  option. 

However,  if  the  health  care  system  is  in  fact 
operating  in  an  era  of  limitations  on  resources 
and  if  the  concern  over  rising  health  care  costs 
continues  to  lead  to  pressures  for  better  balanc- 
ing of  costs  with  outcomes  in  the  use  of  medical 
technology,  this  option  becomes  much  less  at- 
tractive. It  might  eliminate  most  of  the  current 
and  future  developmental  work  on  techniques 
for  comparing  costs  to  benefits.  It  would  force 
reliance  on  current  methods  of  making  deci- 
sions. If  Congress  is  not  satisfied  with  the 
allocation  and  use  of  medical  technology  and 
other  health  care  resources  and  not  satisfied 
with  the  cost  implications  of  such  allocations, 
some  method  for  more  explicitly  forcing  a 
weighing  of  costs  with  benefits  may  need  to  be 
developed.  (Options  7  through  10  below  could 
be  a  step  in  that  direction.) 

Thus,  this  option  should  not  be  adopted  if 
Congress  believes  that  more  explicit  balancing 
of  costs  and  benefits  is  necessary  and  that  CEA/ 
CBA  or  similar  techniques  hold  the  potential  to 
contribute  to  that  process.  Rejection  of  this  op- 
tion does  not  mean  that  immediate  mandatory 
use  of  CEA/CBA  should  take  place;  it  simply 
does  not  shut  the  door  to  that  possibility  or  to 
the  possibility  of  encouraging  methodological 
development,  experiments  with  increased  use  of 
CEA/  CBA,  and  increased  use  of  formal  analyt- 
ical techniques  for  balancing  costs  with  benefits. 
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OPTION  2 

Allow  current  trends  in  development  and  use  of 
CEA/CBA  to  continue,  but  make  no  changes  in 
legislative  requirements  or  prohibitions  relating 
to  the  use  of  CEA/CBA. 

This  is  basically  a  status  quo  option,  but  it 
recognizes  that  the  field  of  CEA/CBA  is  under- 
going methodological  change  and  that  analysts 
and  policymakers  are  paying  increasing  atten- 
tion to  the  potential  uses  of  the  techniques. 
These  trends  would  continue  under  this  option. 
(They  could  be  encouraged  through  implemen- 
tation of  any  or  all  of  Options  7  through  10.) 
But  no  changes  in  legislative  language  regarding 
the  formal,  explicit  use  of  CEA/CBA  in  pro- 
gram decisions  would  be  made.  Since  OTA 
found  that  very  little  formal  use  of  CEA/CBA 
currently  takes  place,  that  nonuse  would  most 
likely  continue  into  the  near  future. 

Under  this  option,  health  care  programs 
would  be  allowed  to  examine  the  possibility  of 
using  more  explicit  costs  and  benefits  informa- 
tion. But  they  would  not  be  required  or  encour- 
aged to  do  so  by  Congress. 

In  medicare  reimbursement  decisions,  this  op- 
tion would  enable  HCFA  and  PHS  to  continue 
their  investigation  into  the  possibility  of  adding 
cost  effectiveness  to  the  list  of  coverage  criteria. 
At  the  same  time,  the  option  recognizes  that  the 
negative  aspects  of  adding  cost  effectiveness  as  a 
criterion  (discussed  below  under  Option  4)  are 
quite  serious.  It  would  allow  demonstration 
projects  (as  presented  in  Option  10)  to  take 
place  before  a  general  decision  to  change  current 
reimbursement  policies  is  made. 

Current  legislative  language  for  market  ap- 
proval of  drugs  and  medical  devices  would  con- 
tinue to  specify  only  safety  and  effectiveness  as 
primary  decision  criteria.  This  situation  seems 
to  reflect  congressional  intent  to  date  and  would 
avoid  the  problems  associated  with  changing 
the  laws  to  include  cost  effectiveness  (as  exam- 
ined in  Option  6  below.) 

Similarly,  other  health  care  programs  would 
be  allowed  to  continue  investigating  the  possi- 
bility of  using  more  explicit  costs  and  benefits 
information.  But  they  would  not  be  required  or 
encouraged  to  do  so  by  Congress.  For  example, 


the  health  planning  program  could  continue  to 
provide  technical  assistance  regarding  cost  and 
cost-effectiveness  analysis  to  local  HSAs.  Also, 
Congress  could  encourage  or  discourage  the  es- 
tablishment and  expansion  of  HMOs  without 
mandating  or  prohibiting  their  use  of  cost-effec- 
tiveness criteria  for  decisionmaking  regarding 
technology. 

Selection  of  this  option  would  in  part  reflect 
the  view  that  the  pros  and  cons  of  the  other  op- 
tions are  too  uncertain  to  permit  a  definite  deci- 
sion on  prohibition  or  on  active  encouragement 
of  use  of  CEA/CBA.  But  it  also  would  reflect  a 
belief  that  some  method  of  more  explicit  con- 
sideration of  costs  and  benefits  is  needed. 

OPTION  3 

Encourage  the  explicit  consideration  of  costs 
and  benefits  in  resource  allocation,  but  do  not 
require  it  as  a  formal  decision  criterion. 

Selection  of  this  option  requires  a  more  favor- 
able view  of  the  current  potential  of  formal 
analysis  than  does  the  previous  option.  This  op- 
tion, in  effect,  says  that  health  care  is  at  the 
point  where  costs  should  be  explicitly  compared 
to  benefits  when  decisions  about  resource  allo- 
cation are  made,  and  that  the  techniques  that 
are  available  to  make  such  comparisons  are  use- 
ful— though  not  well  enough  developed  to  be 
used  as  mandatory  aspects  of  the  decisions.  This 
option  goes  beyond  the  status  quo.  Congress, 
by  adopting  this  approach,  would  be  explicitly 
approving  the  approach  of  CEA/CBA.  Decision 
areas  such  as  health  planning  would  receive 
clear  signals  that  decisions  on  resource  alloca- 
tion should  be  in  part  based  on  formal  iden- 
tification and  (where  possible)  measurement 
and  valuation  of  relevant  costs  and  benefits. 

Should  this  approach  be  taken,  Congress 
could,  for  example,  inform  HCFA  and  PHS  that 
it  approves  of  efforts  examining  the  possibility 
of  including  cost  effectiveness  of  medical  tech- 
nology as  a  criterion  for  reimbursement  cover- 
age and  that  it  approves  use  of  such  a  criterion  if 
that  examination  finds  it  to  be  feasible.  Reim- 
bursement coverage  is  mentioned  here  only  as 
an  example;  the  difficulties  in  this  area  are  sub- 
stantial (see  Option  4)  and  this  may  not  be  an 
area  where  Congress  wishes  to  encourage  CEA/ 
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CBA.  Similar  statements  can  be  made  in  regard 
to  encouraging  the  use  of  CEA/CBA  in  market 
approval  decisions  by  FDA. 

There  are,  however,  areas  where  more  explic- 
it consideration  of  costs  and  benefits  might  be 
encouraged  with  less  negative  impact.  One  is  in 
resource  allocation  decisions  by  health  planning 
agencies.  Costs  and  benefits  are  already  listed  as 
criteria  for  such  decisions.  Also,  quality  assur- 
ance activities  of  PSROs  might  be  modified  to 
take  costs  and  benefits  into  more  explicit  consid- 
eration. Encouragement  in  these  areas  might 
take  the  form  of  providing  increased  analytical 
capabilities  to  agency  staffs,  providing  technical 
assistance  based  on  modified  CEA/CBA-de- 
rived  national  data  for  agency  use,  and,  im- 
portantly, signaling  a  congressional  desire  that 
all  relevant  costs  and  benefits  of  agency  deci- 
sions be  considered,  not  just  those  costs  or  ben- 
efits that  may  be  easily  quantified. 

The  essence  of  this  option  is  that  costs  and 
benefits  should  be  a  part  of  resource  allocation 
decisions,  but  that  current  methodologies  argue 
against  leaning  on  formal  analysis  too  heavily. 

A  positive  aspect  of  this  option  is  its  direct 
recognition  of  the  necessity  of  having  some  way 
of  making  resource  tradeoffs.  The  option  may 
also  result  in  an  increase  in  knowledge  about  the 
value  of  current  and,  eventually  improved,  ana- 
lytical techniques  such  as  CEA/CBA.  A  nega- 
tive aspect  is  the  difficulty  of  encouraging  use  of 
CEA/CBA  while  not  requiring  it.  Substantial 
thought  should  go  into  the  specific  means  of  im- 
plementing this  option  in  individual  health  care 
programs. 

OPTION  4 

Specify  that  medicare  reimbursement  criteria 
should  include  consideration  of  technology's 
cost  effectiveness. 

This  option  would  add  consideration  of  costs, 
which  have  not  been  a  criterion  to  date,  to  the 
largest  Federal  health  program.  The  option's  de- 
sirability depends  on  philosophical,  methodo- 
logical, and  logistical  factors.  Philosophically, 
selection  of  this  option  would  signal  a  change  in 
the  rationale  of  medicare  and  perhaps  other 


Government-funded  health  programs.  Current- 
ly, medicare  operates,  with  some  exceptions,  as 
a  program  to  enable  the  aged  and  disabled  to 
have  access  to  the  medical  services  available  to 
the  general  population.  For  some  very  expensive 
life-prolonging  technologies  (e.g.,  renal  dial- 
ysis), however,  medicare  has  become  the  vehicle 
by  which  those  technologies  are  made  generally 
available  to  all  age  groups. 

If,  under  this  option,  cost-effectiveness  cri- 
teria are  applied  to  all  technologies  to  be  cov- 
ered under  medicare,  the  aged  and  disabled 
might  be  denied  reimbursement  for  use  of  tech- 
nologies that  were  still  available  to  other  in- 
sured populations. 

Further,  major  changes  in  the  procedures  for 
raising  coverage  questions  would  have  to  be 
made.  The  local  contractors  might  have  to  refer 
a  great  many  questions  to  HCFA  for  resolution, 
and  HCFA  would  depend  on  PHS  for  a  large 
volume  of  analysis.  Moreover,  the  data  on 
which  coverage  recommendations  and  decisions 
would  have  to  be  made  do  not  exist  for  many 
technologies.  The  administrative  expense  would 
be  large  and  the  analytical  base  would  have  to 
be  greatly  expanded  at  considerable  cost.  The 
potential  cost  savings  from  more  cost-effective 
use  of  technologies,  if  that  were  brought  about 
by  application  of  CEA/CBA,  would  have  to  be 
compared  to  the  expenses  generated  by  this  op- 
tion. There  is,  however,  no  way  to  estimate  con- 
fidently either  the  probability  or  the  amount  of 
medicare  cost  savings  that  might  result.  This  op- 
tion, if  implemented  in  the  absence  of  an  overall 
system  to  control  expenditures,  would  have  lit- 
tle effect  by  itself  on  the  absolute  amount  of  ex- 
penditures. 

Consideration  might  be  given,  however,  to 
requiring  that  the  possible  addition  of  major 
high-expense  technologies  or  inexpensive  but 
high-volume  technologies  to  medicare  be  care- 
fully assessed  and  their  potential  impact  on  the 
total  health  budget  be  analyzed.  Tradeoffs  be- 
tween increases  in  medicare  expenses  and  in- 
creases in  medicaid  benefits  or  PHS  programs 
could  then  be  considered. 
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OPTION  5 

Require  that  resource  allocation  decisions  by 
health  planning  agencies  be  based  in  part  on  for- 
mal cost-effectiveness  criteria. 

This  option  implies  a  belief  that  the  state-of- 
the-art  of  CEA/CBA  is  sufficiently  advanced  to 
provide  useful  and  valid  information  to  health 
planners.  It  further  implies  that  State  and  local 
health  planning  communities  and  their  constitu- 
ents will  accept  the  results  of  CEA/CBA  studies 
as  inputs  to  the  decision  process.  The  feasibility 
of  the  option  also  depends  on  health  planning 
agencies'  developing  the  capability  to  perform 
or  commission  adequate  analyses  or  having  ac- 
cess to  studies  done  by  others  that  will  be  adapt- 
able to  the  local  situations. 

OTA  finds  that,  although  recent  legislation 
does  indicate  that  Congress  intends  that  health 
benefits  be  weighed  with  costs  in  planning  deci- 
sions and  that  HRA  is  providing  some  guidance 
to  State  and  local  planning  agencies  in  this 
regard,  there  is  no  consensus  as  to  an  adequate 
analytical  method  to  which  health  planners  can 
turn.  At  the  same  time,  however,  OTA  finds 
that  helpful  information  for  many  decisions  can 
be  obtained  by  following  generally  accepted 
principles  of  analysis,  which  include  explicitly 
enumerating  all  the  costs  and  benefits  of  a  given 
course  of  action.  In  general,  health  planning 
agencies  at  present  do  not  have  sufficient  techni- 
cal skills  to  perform  formal  CEA-type  studies. 

OPTION  6 

Modify  the  food  and  drug  laws  to  include  cost 
effectiveness  as  a  market  approval  criterion  for 
drugs  and  medical  devices. 

This  option  might  force  FDA  to  formally 
compare  the  safety,  efficacy,  and  costs  of  a  new 
product  with  those  same  characteristics  of  ex- 
isting technologies.  Such  an  effort  would  re- 
quire an  extensive  amount  of  data,  much  of 
which  does  not  currently  exist.  The  analytical 
capabilities  of  FDA  would  have  to  expand  and 
change  markedly  to  incorporate  the  new  crite- 
rion. The  administrative  and  analytical  de- 
mands of  this  option  would  be  enormous. 

Most  importantly,  the  decision  of  whether  to 
select  this  option  would  hinge  on  current  con- 


gressional intent  regarding  market  approval. 
Past  intent,  as  reflected  in  the  statutes,  explicitly 
mentions  only  safety  and  effectiveness  (effi- 
cacy). Approving  this  option  would  mean  that 
the  intent  of  Congress  regarding  FDA's  regula- 
tory function  has  changed.  If  such  is  not  the 
case,  the  option  is  inadvisable.  If  intent  has 
changed,  then  the  feasibility  of  the  option  de- 
pends on  the  minimization  of  the  administrative 
and  methodological  problems. 

Options  Addressing  the  Techniques  of 
CEA/CBA  Themselves,  or  Their 
Development 

OPTION  7 

Encourage  research  on  the  methods  of  CEA  and 
CBA,  concentrating  on  general  principles  of 
analysis. 

OPTION  8 

Encourage  the  conduct  of  increased  numbers  of 
CEA/CBAs  of  medical  technology. 

OPTION  9 

Encourage  the  development  of  a  strategy  for 
identifying  and  collecting  information  needed 
for  CEA/CBA. 

OPTION  10 
Require  or  encourage  demonstration  projects  to 
test  the  feasibility  of  CEA/CBA  as  an  aid  in 
decisionmaking. 

These  options  are  not  mutually  exclusive;  any 
combination  of  them  could  be  implemented. 
They  are  designed  to  yield  information  about 
the  future  usefulness  of  CEA/CBA  in  decision- 
making and  to  increase  CEA/CBA's  potential 
for  being  useful.  None  of  the  options  requires  an 
immediate  commitment  to  or  decision  on  the  ul- 
timate use  of  CEA/CBA.  All  might  contribute 
to  advancement  of  the  state-of-the-art  of 
CEA/CBA. 

Research  as  proposed  in  Option  7  might  con- 
centrate on  issues  such  as  the  development  of: 
1)  principles  of  formal  analysis,  as  presented 
above,  that  could  be  used  by  analysts  to  con- 
duct studies  and  by  decisionmakers  to  evaluate 
and  use  studies;  2)  "array"  or  nonaggregated 
methods  of  presenting  and  considering  costs  and 
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benefits;  3)  measurement  indexes  that  attempt 
to  capture  and  weigh  divergent  variables;  and 
4)  analytical  approaches  to  various  categories  of 
decisions  in  health  care,  for  example,  the  use  of 
diagnostic  technology,  or  assessment  of  new 
versus  established  technologies.  This  research 
could  be  coordinated  with  the  studies  that 
would  be  conducted  under  Option  8.  Those 
studies,  in  order  to  help  determine  the  potential 
usefulness  of  CEA/CBA,  should  be  undertaken 
or  at  least  designed  and  evaluated  in  collabora- 
tion with  agencies  making  policy  decisions  relat- 
ing to  medical  technology.  Option  8  cannot  be 
merely  an  academic  exercise;  it  cannot  be  done 
in  a  policy  vacuum.  If  it  is,  few  relevant  lessons 
about  CEA/CBA  will  result.  Further,  because 
the  number  of  technologies  that  could  be  studied 
is  extremely  large,  the  setting  of  priorities  for  se- 
lecting those  to  be  analyzed  is  of  critical  impor- 
tance. 

Option  9  does  not  mean  the  collection  of  ad- 
ditional data.  OTA  believes  that  such  collection 
should  be  postponed  until  much  more  thought 
has  gone  into  a  strategy  that  specifies  the  most 
useful  kinds  and  forms  of  information  needed  to 
conduct  analyses  of  costs  and  benefits.  The  use- 
fulness of  CEA/CBA  is  critically  dependent  on 
the  quality  of  the  data  available.  Currently,  the 
state  of  many  types  of  data,  especially  data 
on  efficacy  and  safety  of  medical  technology,  is 
inadequate.  Much  work  has  already  gone  into 


examining  the  state  of  health  data  systems  and 
content.  The  strategy  envisioned  by  Option  9 
would  build  upon  the  existing  studies,  but 
would  require  consideration  of  the  specific 
needs  of  CEA/CBA  analysts  and  decisionmak- 
ers. 

The  need  for  some  method  of  comparing  the 
effects  of  health  care  activities  with  the  re- 
sources consumed  will  remain  critical  to  policy- 
makers. Thus,  the  potential  of  CEA/CBA  or 
some  derivation  should  be  explored  fully.  This 
'may  require,  or  at  least  benefit  from,  limited  ex- 
periments on  the  actual  formal  application  of 
CEA/CBA  in  program  decisionmaking.  Three 
possible  areas  for  the  demonstration  projects  of 
Option  10  may  be  medicare's  reimbursement 
coverage  decisions,  the  resource  allocation  ac- 
tivities of  health  planning  agencies,  and  the  re- 
view activities  of  the  PSROs.  Option  10  recog- 
nizes that  the  methods  of  CEA/CBA  may  im- 
prove, that  data  inadequacies  may  be  lessened, 
and  that  methods  of  assuring  the  appropriate 
consideration  of  CEA/CBA  results  (in  relation 
to  other  variables  of  the  decision)  may  be  devel- 
oped. Thus,  today's  judgments  of  the  role  and 
usefulness  of  CEA/ CBA  may  need  modification 
later.  In  fact,  demonstration  projects  may  help 
to  advance  the  usefulness  of  CEA/CBA  by  con- 
tributing to  advances  in  methods,  data,  and  so 
on. 
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2. 

Evaluating  Costs  and 
Benefits  in  Health  Care 


INTRODUCTION 

A  substantial  share  of  the  health  policy  com- 
munity seems  to  perceive  cost-effectiveness 
analysis/cost-benefit  analysis  (CEA/CBA)  as  a 
potentially  significant  aid  in  the  attempt  to  re- 
duce health  cost  inflation  and  inefficient  re- 
source allocation.  Officials  of  both  public  and 
private  sector  third-party  payers  have  cited 
their  desire  for  CEA/CBA  to  assist  them  in  re- 
imbursement decisions.  Recent  conferences  on 
medical  technology  issues,  attended  by  experts 
from  academia,  Government,  and  the  private 
sector,  have  focused  on  the  need  for  technology 
assessment,  given  its  presumed  significant  po- 
tential role  in  reimbursement  decisionmaking 
(427).  In  addition,  the  Federal  Government  is 
building  the  institutional  machinery  to  guide, 
support,  and  possibly  use  analysis. 

More  generally,  the  health  care  literature  ex- 
hibits a  growing  interest  in  the  cost  effectiveness 
of  health  care.1  Further,  a  wide  variety  of  health 
care  organizations  have  demonstrated  concern 
with  the  resource  allocation  and  cost  contain- 
ment issues  that  CEA/CBA  is  presumed  to  ad- 
dress. Illustrative  of  the  interest  and  concern  are 
growth  in  the  teaching  of  health  economics  in 
medical  school  curricula  (18),  the  separate  ef- 
forts of  the  American  College  of  Physicians  and 
the  Resident  Physicians  Section  of  the  American 
Medical  Association  to  promote  cost-effective 
care,  and  the  work  of  Professional  Standards 
Review  Organizations,  medical  society  cost 
containment  committees,  and  so  on. 

The  potential  of  CEA/CBA  to  contribute  sig- 
nificantly to  cost  containment  and  improved  re- 
source allocation  seems  to  be  an  article  of  faith 
to  many  officials  and  health  policy  experts,  but 
both  the  potential  significance  and  nature  of  any 

'See  Background  Paper  #!.-  Methodological  Issues  and  Litera- 
ture Review,  prepared  by  OTA  in  conjunction  with  this  assess- 
ment. 


contributions  of  these  techniques  remain  to  be 
established.  Despite  the  fact  that  different 
observers  have  different  kinds  of  impacts  in 
mind  when  they  discuss  these  analytical  tech- 
niques, the  several  qualitatively  distinct  kinds  of 
impacts  CEA/CBA  might  have  are  rarely  articu- 
lated explicitly. 

The  purpose  of  this  chapter  is  to  set  the  stage 
for  later  discussions  of  findings  regarding  the 
usefulness  or  potential  usefulness  of  CEA/CBA. 

The  public,  or  governmental,2  sector  is  called 
upon  to  make  certain  decisions  that  are  imprac- 
tical for  the  private  sector  to  make.  Examples  of 
these  are  decisions  concerning  national  defense 
or  air  pollution  control,  neither  of  which  is 
amenable  to  being  traded  in  the  marketplace. 
The  public  sector  also  must  make  other  deci- 
sions for  social  reasons  such  as  assuring  equit- 
able distribution  of  what  may  be  considered  es- 
sential goods  (e.g.,  health  care  of  the  elderly). 
Although  the  inherent  complexities  of  these  de- 
cisions and  the  uncertainties  of  the  variables  in- 
volved make  it  extremely  difficult  to  identify 
and  weigh  all  the  consequences,  the  quality  and 
validity  of  the  decisions  can  be  greatly  enhanced 
by  structuring  the  process  so  as  to  reveal  as 
many  of  the  relevant  costs  and  benefits  as  is  fea- 
sible and  by  evaluating  them  from  a  social  per- 
spective. 

Since  conventional  private  sector  techniques 
such  as  capital  budgeting  and  return-on-invest- 
ment  analysis  are  ordinarily  insufficient  for  de- 
cisions made  in  the  public  arena,  special  tech- 


2This  report  uses  the  term  "public  sector"  to  refer  to  governmen- 
tal institutions  and  the  term  "private  sector"  to  refer  to  all  other 
segments  of  society.  Some  people  further  distinguish  between  the 
for-profit  "private"  sector  and  the  not-for-profit  "private"  sector 
(including,  e.g.,  foundations,  universities,  or  institutions  such  as 
the  Blue  Cross  Association).  The  term  "independent  sector"  has 
been  suggested  for  this  latter  sector. 
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niques  have  been  developed.  In  the  medical  care 
area,  two  such  techniques  are  CEA  and  CBA. 
CEA  and  CBA  are  designed  to  integrate  the  eco- 
nomic aspects  of  a  decision  with  the  health  as- 


pects of  that  decision.  Consequently,  they 
should  not  be  considered  simply  as  economic 
tools. 


DEFINITION  OF  CEA/CBA 

The  terms  CEA  and  CBA  have  come  to  refer 
to  formal  analytical  techniques  for  comparing 
the  positive  and  negative  consequences  of  alter- 
native ways  to  allocate  resources.  In  practice, 
the  comparison  of  costs  and  benefits  is  accom- 
plished through  a  spectrum  of  approaches, 
ranging  from  sophisticated  computer-based 
mathematical  programing  using  large  amounts 
of  epidemiological  and  other  data  to  partially 
intuitive,  best-guess  estimates  of  costs  and  bene- 
fits. Some  analyses  may  take  into  account  the 
results  of  clinical  trials  of  a  technology  and 
model  the  technology's  effect  on  health  out- 
comes. Others  may  assume  that  the  alternative 
technologies  under  study  have  equal  effective- 
ness and  concentrate  on  the  difference  in  costs 
involved. 

Thus,  there  is  a  continuum  of  analyses  that 
examine  costs  and  benefits.  At  one  end  of  the 
continuum  are  what  will  be  referred  to  as  "net 
cost"  studies.  In  these  studies,  the  emphasis  is  on 
costs;  net  cost  studies  in  the  past  have  often  as- 
sumed benefits  or  efficacy  to  be  equal.  At  the 
other  end  of  the  continuum  are  analyses  that  at- 
tempt to  relate  the  use  of  the  technologies  under 
study  to  specific  health-related  outcomes  and 
compare  the  costs  of  the  technologies  to  the  dif- 
ferential health  benefits.  Thus,  CEA/CBA  com- 
prises a  set  of  analytical  techniques,  differenti- 
ated by  the  specific  costs  and  benefits  that  are 
considered  and  the  manner  in  which  they  are 
analyzed. 

The  principal  distinctions  between  CEA  and 
CBA  lie:  1)  in  the  method  of  valuation  of  the 
desirable  consequences  of  a  decision,  and  2)  in 
the  implications  of  the  different  methods  of  that 
valuation.  In  CBA,  all  costs  and  all  benefits  are 
valued  in  monetary  terms.  In  a  CBA  (simplified 
for  illustration)  of  an  emergency  medical  sys- 
tem, for  example,  the  cost  of  the  program  would 


be  estimated  and  compared  to  the  benefits.  The 
latter,  which  might  consist  of  saving  a  certain 
number  of  lives,  would  somehow  be  valued  in 
dollars,  and  the  results  of  the  analysis  would  be 
.expressed  in  dollar  cost  per  dollar  benefit.  Thus, 
conceptually,  CBA  can  be  used  to  evaluate  the 
"worth"  of  a  project  and  would  allow  compari- 
son of  projects  of  different  types  (e.g.,  such  as 
dams  and  hospitals). 

In  CEA,  on  the  other  hand,  desirable  program 
consequences  are  not  valued  in  monetary  terms, 
but  rather  are  measured  in  some  other  unit; 
common  measures  include  years  of  life  saved, 
days  of  morbidity  or  disability  avoided,  or  com- 
bination of  morbidity  and  mortality  such  as 
quality-adjusted  life  years  (QALYs)  saved.  The 
reason  for  a  nonmonetary  measure  of  program 
effectiveness  is  either  the  impossibility  or  the 
undesirability  of  valuing  certain  outcomes  in 
dollars.  Thus,  unlike  that  of  a  CBA,  the  "bot- 
tom line"  of  a  CEA  is  not  expressed  as  a  net 
monetary  value  or  a  monetary  ratio;  rather,  it  is 
expressed  in  units  such  as  "dollars  per  life 
saved"  or  "dollars  per  QALY  gained."  CEA  does 
permit  comparison  of  cost  per  unit  of  effective- 
ness among  competing  program  alternatives  de- 
signed to  serve  the  same  basic  purpose.  Unlike 
CBA,  however,  it  does  not  allow  comparison  of 
programs  having  vastly  different  objectives  (be- 
cause the  effectiveness  or  outcome  measures  dif- 
fer), nor  does  it  permit  assessment  of  the  inher- 
ent worth  of  a  program.  Is  a  cost  of  $50,000  per 
year  of  life  saved  acceptable?  Obviously,  this 
question  requires  a  social  and  political  judg- 
ment; it  is  not  simply  a  technical  matter.  Natu- 
rally enough,  as  the  state-of-the-art  of  devel- 
oping composite  indexes  of  outcome  measures 
improves,  the  versatility  of  CEA  also  advances, 
because  the  technique  can  then  be  used  to  com- 
pare increasingly  divergent  programs. 
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In  this  report,  the  term  CEA/CBA  is  used  to  CEA  and  CBA.  The  findings  of  the  following 
refer  to  the  class  of  techniques  that  includes  both       chapters  apply  generally  to  both  techniques. 


IMPORTANCE  OF  EFFICACY  AND  SAFETY  DATA 


As  emphasized  above,  CEA/CBA  should  not 
be  considered  only  an  economic  tool.  This  point 
is  not  negated  by  the  fact  that  CEA/CBA  is  de- 
scribed as  an  efficiency-based  technique.  Meas- 
urement of  efficiency  depends  as  much  on  out- 
put as  on  resources  used  to  produce  the  output. 
OTA  believes  that  one  of  the  critical  output  or 
outcome  measures  that  are  or  can  be  addressed 
by  CEA/CBAs  is  effect  on  health  status  or  other 
health-outcome-related  effects.  Any  CEA/CBA 


that  attempts  to  analyze  such  outcomes  for  eval- 
uation of  medical  technology  will  be  only  as 
comprehensive  and  valid  as  the  data  on  the  ef- 
ficacy and  safety  of  that  technology  are.  Thus, 
health-outcome-related  CEA/CBAs  are  depend- 
ent on  the  existence  of  an  adequate  efficacy  and 
safety  information  base.  As  OTA  has  reported 
previously,  however,  the  status  of  such  infor- 
mation is  inadequate  for  many  medical  technol- 
ogies (465). 


HISTORY  OF  CEA/CBA 

The  commonsense  principles  of  CEA/CBA 
have  been  promoted  for  centuries,  but  formal 
application  of  CEA/CBA  is  primarily  a  phe- 
nomenon of  the  present  century.  In  1902,  the 
River  and  Harbor  Act  directed  the  Corps  of  En- 
gineers to  assess  the  costs  and  benefits  of  river 
and  harbor  projects.  In  1936,  the  Federal  Flood 
Control  Act  required  that  "the  benefit  (of  proj- 
ects) to  whomsoever  they  may  accrue  (must  be) 
in  excess  of  the  estimated  costs,"  though  the  Act 
provided  no  guidance  as  to  how  benefit  and 
costs  were  to  be  defined  and  measured.  In  the 
same  decade,  both  the  Tennessee  Valley  Au- 
thority and  the  Department  of  Agriculture  im- 
plemented program  budgeting  systems  that  in- 
cluded rudimentary  attempts  at  formal  CEA/ 
CBA.  Official  Government  criteria  for  appraisal 
of  river  development  projects  were  first  enunci- 
ated by  the  Bureau  of  the  Budget  in  1952  (585). 

Early  in  the  Kennedy  administration,  the  De- 
fense Department,  under  Secretary  McNamara, 
adopted  a  program  budgeting  system  that  em- 
ployed CEA/CBA  to  evaluate  alternative  de- 
fense projects.  Success  in  these  endeavors,  com- 
bined with  a  burgeoning  Federal  budget,  led 
President  Johnson  in  1965  to  require  the  imple- 
mentation of  planning-programing-budgeting 
(PPB)  systems  throughout  the  Federal  Govern- 
ment. CEA/CBA  represented  both  the  spirit  and 


the  letter  of  the  new  initiative  to  rationalize 
Government  resource  allocation  decisionmak- 
ing (547). 

PPB  met  with  mixed  and  limited  success,  re- 
flecting a  lack  of  resources  to  implement  it  effec- 
tively, political  and  bureaucratic  opposition  to 
it,  and  unrealistic  expectations  of  its  role  and 
potential  (256,  352).  The  formal  system  did  not 
survive  for  long,  though  many  Washington  ob- 
servers believe  it  left  a  legacy  of  continuing  im- 
provement in  the  use  of  rational  analysis  in 
Government  decisionmaking  (516).  The  philo- 
sophy and  logic  of  CEA/CBA  and  PPB  have 
been  reincarnated  during  the  Carter  administra- 
tion in  the  form  of  zero-based  budgeting. 

In  this  assessment,  OTA  found  relatively  few 
sound  applications  of  CEA/CBA  in  health  re- 
source decisionmaking.  There  are,  however,  in- 
creasing numbers  of  such  studies  being  per- 
formed, particularly  in  the  applied  research 
field,  and  the  results  of  these  studies  are  increas- 
ingly being  disseminated.  Although  it  is  difficult 
to  know  how  much  effect  this  type  of  informa- 
tion has,  there  is  ample  evidence  that  both  the 
private  and  public  health  sectors  are  increasing- 
ly cost  conscious.3 

3See  ch.  2  of  Background  Paper  #1:  Methodological  Issues  and 
Literature  Review,  prepared  by  OTA. 
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OTA  found  two  divergent  trends  in  published 
CEA/CBA  studies.  One  trend,  based  largely 
within  the  academic  community  but  slowly 
diffusing  to  research-oriented  practitioners,  is 
leading  to  rapid  changes  in  the  state-of-the-art 
of  CEA/CBA.  This  trend  is  toward  CEA/CBA 
studies  characterized  as  interdisciplinary,  of 
high  quality,  and  advancing  the  state-of-the-art 
of  the  methods.  These  studies  are  also  becoming 
increasingly  oriented  to  concerns  peculiar  to  the 
health  field,  such  as  health  status  measurement, 
equity  considerations,  and  the  value  of  health 
and  of  life  itself.  Since  the  studies  are  gaining 
wide  readership — being  published  in  the  leading 
journals — and  are  becoming  more  understand- 
able to  the  lay  public,  they  are  enhancing  ac- 
ceptance of  CEA/CBA. 


The  other  trend,  which  is  occurring  outside 
the  academic  community,  is  characterized  by  a 
rapidly  expanding  CEA/CBA  literature  base. 
Many  of  the  articles  are  written  by  practitioners 
who  are  increasingly  concerned  about  the  gene- 
ral concept  of  cost  effectiveness  in  medical  prac- 
tice. For  the  most  part,  this  body  of  literature  is 
found  in  general  health  and  medical  journals. 
Although  the  analyses  are  not  as  methodologi- 
cally advanced  or  complex  as  those  in  the  form- 
er group — probably  because  the  authors  often 
do  not  have  an  economic  or  other  quantitative 
background — their  impact  upon  physician  prac- 
tice may  be  substantial.  The  increasing  number 
of  studies,  in  any  case,  certainly  can  be  consid- 
ered an  index  of  practitioners'  concerns  about 
health  costs. 


LEGAL  STATUS  OF  CEA/CBA 

Currently,  the  law  explicitly  authorizes  only 
one  health-care-related  agency,  the  National 
Center  for  Health  Care  Technology,  to  support 
CEA/CBA  studies,  although  parts  of  the  Na- 
tional Health  Planning  and  Resource  Develop- 
ment Act  (Public  Law  93-641)  require  that  cost 
effectiveness  be  considered  in  some  decision 
processes.  Furthermore,  no  court  rulings  or 
pending  cases  directly  relate  to  the  use  of  these 
techniques  in  health  care  agencies.  In  areas 


other  than  health  (e.g.,  environmental  regula- 
tion), however,  there  are  immediate  and  signifi- 
cant judicial  and  legislative  pressures  to  use 
CEA/CBA  in  decisionmaking.  In  the  last  two 
Congresses  alone,  more  than  65  separate  bills 
have  included  provisions  for  Federal  agencies  to 
use  CEA/CBA  or  risk-benefit  analysis  in  the  de- 
cisionmaking process.  See  appendix  E  for  a  brief 
discussion  of  the  legal  status  of  CEA/CBA. 


USE  OF  CEA/CBA  IN  HEALTH  CARE  DECISIONMAKING 


There  are  two  basic  types  of  health  care  re- 
source allocation  decisions  that  in  theory  could 
benefit  from  CEA/CBA.  The  first  are  decisions 
made  within  a  fixed,  constrained,  or  popula- 
tion-based budget,  such  as  those  made  by  health 
maintenance  organizations  (HMOs).  The  sec- 
ond are  decisions  made  in  the  absence  of  a  direct 
budget  constraint,  such  as  reimbursement  deci- 
sions by  medicare  or  certificate-of-need  recom- 
mendations by  local  health  planning  agencies. 

In  the  former — allocation  decisions  made 
within  a  budget — tradeoffs  must  be  made,  since 
not  all  projects  can  be  funded.  The  projects  that 
promise  to  deliver  more  benefits  for  the  cost 


should  be  more  attractive  than  those  projects 
expected  to  deliver  fewer  benefits.  In  these  deci- 
sions, an  economic  constraint  already  forces 
costs  to  be  considered.  The  function  of  CEA/ 
CBA  in  budget-constrained  decisions,  therefore, 
would  be  to  illuminate  the  decision  process  and 
to  require  that  implicit  judgments  be  made  more 
explicitly,  thus  forcing  external  examination  of 
assumptions  and  of  the  values  placed  on  the  de- 
cision variables.  Note  that  even  though  costs  are 
normally  taken  into  account  in  these  budget- 
constrained  situations,  the  types  of  CEA/CBA 
or  related  analyses  undertaken  can  still  range 
from  analyses  on  the  net  cost  end  of  the  con- 
tinuum to  analyses  where  effectiveness  is  ex- 
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plicitly  related  to  health  outcomes  or  some 
equivalent  measure. 

In  non-budget-constrained  decisions,  direct 
tradeoffs  between  competing  projects  often  are 
unnecessary.  Consequently,  a  function  of  CEA/ 
CBA  in  these  decisions  would  be  to  force  eco- 
nomic factors  to  be  considered.  In  health  plan- 
ning decisions,  for  example,  planners  would  be 
asked  not  only  to  consider  whether  a  service  is 
needed  but  also  to  compare  the  cost  of  the  serv- 
ice with  the  expected  benefits  and  perhaps  to 
compare  the  costs  and  benefits  of  the  service 
under  study  to  the  costs  and  benefits  of  other 
services  that  could  be  assigned  higher  or  lower 
priorities. 

CEA/CBA  is  viewed  by  different  parties  as 
ranging  in  usefulness  from  obfuscating  the  perti- 
nent issues  in  a  decision  process  at  one  extreme 
to  illuminating  and  synthesizing  the  issues  so 
well  that  the  technique  is  used  to  make  decisions 
at  the  other  extreme.  There  is,  however,  a  mid- 
dle position  that  maintains  that  the  technique 
could  be  helpful  in  structuring  information  and 
that  this  information  should  be  only  one  of 
several  components  of  a  decision  process.  Both 
extreme  positions  mentioned  above  are  associ- 
ated with  the  use  of  CEA/CBA  as  a  formal, 
structured  analysis  that  is  oriented  toward  a 
bottom-line  answer,  such  as  a  cost-benefit  ratio. 
Such  a  bottom-line,  however,  may  avoid  or 
even  hide  many  important  value  judgments, 
thus  providing  an  unambiguous  answer  which 
may  rest  on  ambiguous  data  or  assumptions. 


Advocates  of  the  middle  position  propose  that 
CEA/CBA  be  used  within  the  context  of  ac- 
cepted principles  of  analysis  in  order  to  il- 
luminate the  costs  and  the  benefits  of  a  decision, 
but  not  necessarily  to  aggregate  and  weigh 
them.  Warner  mentions  a  similar  perception  of 
CEA/CBA  as  a  consciousness-raising  exercise: 
CEA/CBA  would  have  "no  direct  influence  on 
policy  decisions,  but  its  presence  in  the  litera- 
ture and  in  policy  debates  (would  serve)  to  raise 
the  general  awareness  and  understanding  of  the 
economic  side  of  health  care,  particularly 
among  members  of  the  medical  profession" 
(615).  When  properly  conducted,  CEA/CBA 
can  serve  as  a  means  of  raising  value  and  equity 
issues  related  to  the  subjects  under  study. 

Most  of  the  specific  findings  of  this  report  re- 
late to  two  major  general  findings  of  the  OTA 
assessment.  The  first  of  the  general  findings  is 
that  performing  an  analysis  of  costs  and  benefits 
has  the  potential  to  be  very  helpful  to  decision- 
makers, because  the  process  of  analysis  struc- 
tures the  problem,  allows  an  open  consideration 
of  all  relevant  effects  of  a  decision,  and  forces 
the  explicit  treatment  of  key  assumptions.  The 
second  general  finding  is  that  CEA/CBA  ex- 
hibits too  many  methodological  and  other  limi- 
tations to  justify  relying  solely  or  too  heavily  on 
the  results  of  formal  CEA/CBA  studies  in  mak- 
ing a  decision.  Thus,  CEA/CBA  could  be  useful 
for  assisting  in  many  decisions,  but  is  probably 
not  appropriate  as  the  sole  or  prime  determinant 
of  a  decision. 
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INTRODUCTION 

OTA  found  no  consensus  among  analysts  and 
practitioners  as  to  a  standard  set  of  methods 
for  cost-effective  analysis/cost-benefit  analysis 
(CEA/CBA).  Although  there  is  still  some  dis- 
agreement as  to  which  variables  should  be  con- 
sidered and  how  these  variables  should  be 
treated — problems  which  may  be  lessened  as  the 
state-of-the-art  develops — there  is  agreement 
that  at  present  no  one  method  is  appropriate  for 
any  two  classes  of  technologies  or  for  any  two 
situations  under  which  a  technology  is  being  as- 
sessed. In  general,  the  disagreement  on  precise 
methods  is  due  more  to  the  inherent  nature  of 
the  analysis,  the  nature  and  stage  of  develop- 
ment of  the  technology  being  analyzed,  and  the 
general  social  and  political  environment  of  deci- 
sionmaking than  to  the  immaturity  of  CEA/ 
CBA  methods.  OTA  believes  that  the  funda- 
mental approach  to  CEA/CBA  should  be  based 
on  clear,  logical  thinking,  using  explicit  criteria 
within  the  framework  of  generally  accepted 
methodological  principles. 


OTA  did  find  full  agreement  on  the  paucity 
of,  and  consequently  the  need  for,  improved 
data,  without  which  good  analyses  are  impossi- 
ble. Efficacy  and  safety  information  for  many 
technologies  is  generally  not  available.  Health 
care  utilization  data  are  often  either  not  avail- 
able or  not  in  standard  format  or  accessible 
form,  and  cost  data  are  often  inaccurate  and 
also  nonstandardized.  Better  routine  data  col- 
lection, although  desirable  and  possibly  neces- 
sary, would  probably  not  be  sufficient  for  better 
analyses,  however,  because  each  specific  anal- 
ysis often  requires  a  unique  data  set  that  will  not 
be  available  in  even  the  best  of  routine  data  col- 
lection systems.  Consequently,  an  optimum  mix 
of  routine  data  collection  and  study-specific 
data  collection  needs  to  be  defined,  and  when 
specific  studies  are  funded,  monies  for  necessary 
data  collection  should  be  included. 


METHODOLOGICAL  LIMITATIONS 

The  methodological  weaknesses  or  shortcom- 
ings of  CEA/CBA  are  of  two  general  types:  1) 
those  that  are  inherent  in  this  form  of  analysis, 
and  2)  those  that  are  due  to  the  lack  of  maturity 
in  the  state-of-the-art  of  CEA/CBA  and  to  the 
lack  of  analyst  expertise  and  experience  with 
CEA/CBA  in  health  care.  The  latter  type  can  be 
expected  to  diminish  as  more  experience  accu- 
mulates. The  10  principles  for  analysis  pre- 
sented later  in  this  chapter  are  directly  relevant 
to  lessening  this  type  of  shortcoming,  which  will 
be  called  "weaknesses  due  to  immaturity."  Limi- 
tations of  the  first  type,  those  that  are  inherent, 
however,  are  likely  to  remain  significant  bar- 
riers to  advances  in  the  usefulness  of  CEA/CBA 
in  health  care. 


Weaknesses  Inherent  in  CEA/CBA 

Inherent  weaknesses  of  the  methodology  of 
CEA/CBA,1  especially  in  its  more  formal  or  so- 
phisticated forms,  may  prove  to  be  of  greater 
long-run  consequence  than  those  of  immaturity 
(617). 

Examples  of  such  weaknesses  are  the  difficul- 
ty of  predicting  with  precision  the  costs  and 
benefits  of  new  or  not  yet  existing  programs  or 
technologies,  fundamental  problems  in  quanti- 
fying or  valuing  certain  important  but  less  tan- 

'Inherent  methodological  weaknesses  of  CEA/CBA  are  dis- 
cussed in  detail  in  Background  Paper  #1;  Methodological  Issues 
and  Literature  Review,  prepared  by  OTA  in  conjunction  with  this 
assessment. 
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gible  health  benefits,  controversy  over  the  ap- 
propriate discount  rate,  the  inability  of  analysis 
to  adequately  incorporate  equity  and  political 
considerations,  and  the  inevitability  of  signifi- 
cant sensitivities  or  uncertainties  even  in  many 
perfectly  managed  studies.  The  rapidity  and 
profundity  of  technical  change  in  medicine  exac- 
erbate analytical  difficulties,  a  problem  that  is 
felt  particularly  acutely  because  the  point  at 
which  an  analysis  might  have  the  most  signifi- 
cant impact  on  health  resource  allocation — be- 
fore a  technology  has  diffused  into  widespread 
medical  practice — is  also  the  point  at  which 
evaluation  uncertainties  are  most  dramatic. 
Sensitivity  analysis  sometimes  can  demonstrate 
that  inherent  technical  analytical  problems  do 
not  affect  qualitative  conclusions,  but  frequent- 
ly these  difficulties  preclude  a  definitive  assess- 
ment of  the  desirability  of  competing  programs. 
Ultimately,  research  may  resolve  some  current- 
ly intractable  problems,  but  for  the  foreseeable 
future,  most  such  limitations  seem  likely  to  re- 
main inherent  barriers  to  the  direct  application 
of  the  findings  of  many  CEA/CBAs  to  policy 
decisionmaking.  In  particular,  the  uncertainties 
which  pervade  analysis  severely  restrict  the  po- 
tential of  a  study,  however  high  quality,  to  re- 
solve definitively  the  "close  calls"  in  which  alter- 
native programs  are  similar  in  both  cost  and  ef- 
fectiveness. 


Weaknesses  Due  to  Immaturity 

As  noted  in  the  previous  chapter,  there  are 
relatively  few  examples  of  technically  high- 
quality  CEA/CBA  studies  in  the  health  litera- 
ture. As  the  state-of-the-art  of  CEA/CBA  ma- 
tures and  as  analysts  and  decisionmakers  gain 
more  experience  with  CEA/CBA  in  health  care, 
however,  there  may  be  a  reduction  in  the  num- 
ber of  problems  such  as  inappropriate  or  inaccu- 
rate specification  of  production  relationships;2 
inadequate  identification,  measurement,  or  val- 
uation of  costs  or  benefits;  lack  of  discounting 
•  of  future  costs  and  benefits;  and  failure  to  exam- 
ine sensitivities.  Though  one  should  never 
downplay  the  difficulty  of  producing  a  techni- 
cally high-quality  study,  in  principle  problems 
such  as  these  can  be  resolved;  clearly  the  prac- 
tice of  analysis  can  and  should  improve  over 
time.  Thus,  the  current  restriction  on  the  useful- 
ness of  CEA/CBA  caused  by  weaknesses  of  im- 
maturity seems  likely  to  recede  in  importance 
over  time.  The  10  principles  of  analysis  pre- 
sented below  are  suggested  as  one  method  of 
minimizing  these  weaknesses. 


2See  Background  Paper  #2:  Methodological  Issues  and  Litera- 
ture Review. 


TEN  PRINCIPLES  OF  CEA/CBA  METHODOLOGY 


There  is  widespread  agreement  that  the  10 
basic  principles  below  are  generally  applicable 
to  CEA/CBA  analysis.  (See  table  1.) 

1.  Define  Problem 

The  problem  should  be  clearly  and  explicitly 
defined  and  the  relationship  to  health  outcome 
or  health  status  should  be  stated.  The  problem, 
for  example,  may  be  expressed  terms  such  as 
"excess  infection  rate"  or  "excess  deaths."  The 
broader  the  definition  of  the  problem,  the  more 
relevant  alternatives  there  are  to  examine.  For 
instance,  "excess  deaths"  could  lead  to  compar- 
ing any  preventive  or  therapeutic  program  that 
decreases  mortality,  whereas  "excess  deaths  due 
to  cancer"  would  limit  the  scope  of  study  con- 


siderably, and  "excess  deaths  due  to  cervical 
cancer"  would  limit  it  even  further.  Neverthe- 
less, whatever  the  scope,  as  long  as  the  focus  is 
on  a  health  problem,  the  study  can  focus  on  al- 
ternative means  to  solve  the  problem  or,  con- 
versely, to  increase  or  improve  health  status. 
Some  studies,  however,  must  necessarily  focus 
on  the  efficient  use  of  a  technology.  This  is  par- 
ticularly true  of  studies  of  diagnostic  technol- 
ogies, where  the  ultimate  health  problem  may 
be  far  removed  from  the  use  of  the  technology. 

2.  State  Objectives 

The  objectives  of  the  technology  being  as- 
sessed should  be  explicitly  stated,  and  the  anal- 
ysis should  address  the  degree  to  which  the  ob- 
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Table  1.— Ten  General  Principles  of  Analysis 
(for  CEA/CBA  methodology) 

1.  Define  problem. 

2.  State  objectives. 

3.  Identify  alternatives. 

4.  Analyze  benefits/effects. 

5.  Analyze  costs. 

6.  Differentiate  perspective  of  analysis. 

7.  Perform  discounting. 

8.  Analyze  uncertainties. 

9.  Address  ethical  issues. 
10.  Interpret  results. 

SOURCE:  Office  of  Technology  Assessment. 

jectives  are  (expected  to  be)  met.  In  general,  the 
objectives  will  be  governed  by  the  way  in  which 
the  problem  is  defined;  the  broader  the  problem 
definition,  the  broader  the  objectives  Ordinari- 
ly, it  is  most  relevant  for  the  objectives  to  be  in 
terms  of  lowering  morbidity,  disability,  or  mor- 
tality or,  alternatively,  increasing  well-being. 
When  the  objectives  are  stated  in  terms  of  de- 
creasing costs,  the  relationship  between  costs 
and  health  benefits  is  often  lost,  sometimes  re- 
sulting in  untenable  assumptions  of  equal  ef- 
ficacy across  treatment  modalities.  Often,  ob- 
jectives are  stated  in  terms  of  achieving  a  certain 
level  of  benefit  for  the  least  cost,  or,  conversely, 
achieving  the  most  benefit  per  dollar  cost. 

3.  Identify  Alternatives 

Alternative  means  (technologies)  to  accom- 
plish the  objectives  should  be  identified  and  sub- 
jected to  analysis.  The  number  of  alternatives 
and  the  relevancy  of  the  analysis  will  increase  as 
the  scope  of  the  identified  problem  is  increased. 
Whereas  there  are  numerous  means  to  lower 
death  rates,  for  example,  there  are  relatively 
fewer  ways  to  lower  deaths  due  to  a  specific 
disease,  and  even  fewer  ways  to  lower  these 
deaths  by  employing  a  particular  technology. 
One  of  the  most  difficult  questions  to  answer  in 
analyzing  the  cost  effectiveness  of  a  given  inter- 
vention (such  as  Pap  screening)  is  "cost  effective 
compared  to  what?" 


4.  Analyze  Benefits/Effects 

All  foreseeable  benefits /effects  should  be 
identified  and  when  possible  should  be  meas- 
ured. The  relevant  effects  of  health  care  technol- 
ogy in  the  health  field  often  follow  directly  from 
the  problem  under  consideration,  the  objectives 
specified,  and  the  framework  in  which  the  prob- 
lem is  approached.  Not  all  benefits  or  effects  are 
positive — some  may  be  negative  (e.g.,  deaths 
due  to  surgery)  and  some  may  be  indeterminate 
(e.g.,  incurable  disease  may  be  discovered). 
Each  of  the  following  categories  should  be  con- 
sidered: 1)  personal  benefits/effects,  such  as 
alleviated  pain,  reduced  risk  of  sickness  or 
death,  enhanced  quality  of  life,  lowered  anxiety, 
2)  health  resource  benefits/effects  such  as  in- 
creases and  decreases  in  health  care  expendi- 
tures, 3)  other  economic  benefits/effects  such  as 
increased  productivity,  and  4)  social  benefits/ 
effects  such  as  the  equitable  distribution  of  med- 
ical care.  When  possible,  and  if  agreement  can 
be  reached,  it  is  helpful  to  value  benefits  in  com- 
mon terms  in  order  to  make  comparisons  across 
alternative  programs  easier. 

5.  Analyze  Costs 

All  expected  costs  should  be  identified  and 
when  possible  should  be  measured  in  dollars.  In 
general,  the  concept  of  "opportunity  cost"  is  the 
most  correct  way  to  consider  the  costs  of  a  pro- 
gram. That  is,  the  costs  are  equal  to  the  value  of 
the  opportunities  that  are  forgone  because  of  the 
investment  in  the  program. 

6.  Differentiate  Perspective  of  Analysis 

When  private  benefits  and  costs  differ  sub- 
stantially from  social  benefits  and  costs,  and  if  a 
private  perspective  is  appropriate  for  the  anal- 
ysis, the  differences  should  be  identified.  Al- 
though CEA/CBA  is  generally  considered  a  tool 
of  social  policy,  it  is  helpful  and  important  to 
recognize  that  private  incentives  differ  from 
public  incentives  and  since  health  care  delivery 
is  often  funded,  always  demanded,  and  usually 
delivered  by  the  private  sector,  its  (the  private 
sector's)  perspective  may  be  very  important  to 
the  relevancy  of  the  analysis.  For  instance,  the 
social  benefits  of  elective  procedures  such  as 
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elective  hysterectomy,  cancer  screening,  and 

g  many  psychotherapy  programs  are  apt  to  differ 

p  markedly  from  the  private  benefits.  Typically,  a 

t<  CEA  will  identify  the  "social"  benefits  in  terms 

c  of  cost  reduction;  the  primary  private  objective 

c  (i.e.,  expected  benefits)  of  the  patient,  however, 

p  may  be  decreased  anxiety. 

F 

e    7.  Perform  Discounting 

v        All  future  costs  and  benefits  should  be  dis- 

c  counted  to  their  present  value  in  order  for  them 

f  to  be  compared  with  one  another.  Discounting 

r  can  be  thought  of  as  a  reverse  interest  rate.  It  is 

e  used  to  take  into  account  phenomena  such  as 

J  the  observation  that,  all  things  being  equal,  peo- 

t  pie  prefer  benefits  (including  health  benefits)  to- 

r  day  rather  than  at  a  future  time. 
1 

j  8.  Analyze  Uncertainties 

1        Key  variables  should  be  analyzed  as  to  the 
i     importance  of  their  uncertainty  to  the  results  of 
i     the  analysis.  That  is,  a  "sensitivity  analysis" 
\     should  be  performed.  In  its  simplest  form,  sen- 
i     sitivity  analysis  is  nothing  more  nor  less  than 
the  application  of  commonsense  when  one  is  not 
sure  of  a  fact;  it  is  the  examination  of  the  uncer- 
!     tain  event  under  different  assumptions.  Sensi- 
tivity analysis  can  indicate  both  when  more  in- 
formation is  needed  and  when  insufficient  infor- 
mation is  irrelevant. 


9.  Address  Ethical  Issues 

Ethical  issues  should  be  identified,  discussed, 
and  placed  in  appropriate  perspective  relative  to 
the  rest  of  the  analysis  and  the  objectives  of  the 
technology.  Many  health  care  programs  have  as 
their  primary  objective  the  equitable  distribu- 
tion of  services;  other  programs  include  it  as  one 
of  many  objectives;  still  other  programs  affect 
the  distribution  of  society's  goods  and  services 
without  an  explicit  intention  to  do  so.  A  CEA/ 
CBA  should  identify  all  these  effects.  When  pos- 
sible, it  should  also  measure  them.  Although 
such  effects  cannot  ordinarily  be  valued,  how- 
ever, they  are  often  germane,  and  sometimes 
essential,  to  the  measure  of  worth  of  a  health 
program. 

10.  Interpret  Results 

The  results  of  the  analysis  should  be  discussed 
in  terms  of  validity,  sensitivity  to  changes  in 
assumptions,  and  implications  for  policy  of  de- 
cisionmaking. This  is  important  both  because 
the  intended  audience  is  often  a  public  official  or 
a  health  care  professional,  neither  of  whom  may 
be  technically  oriented,  and  because  study  find- 
ings are  often  reported  in  capsule  form  such  as  a 
news  brief,  and  are  often  introduced  in  the  pro- 
fessional literature  in  abstract  form.  Results  of 
CEA/CBA  often  have  the  potential  to  mislead 
the  reader,  a  hazard  which  can  be  greatly  re- 
duced by  interpretation. 


OTHER  FINDINGS 

In  addition  to  conforming  to  the  aforemen- 
tioned 10  general  principles,  all  quantitative 
analyses  should  specify  data  sources,  be  written 
as  clearly  and  as  nontechnically  as  possible,  and 
be  subjected  to  peer  and  other  types  of  review, 
including  public  scrutiny  when  appropriate, 
especially  regarding  assumptions  upon  which 
the  outcome  of  the  analyses  may  rest.  In  gen- 
eral, the  more  technical  the  analysis,  the  more 
important  that  the  review  be  formalized  and 
conducted  by  individuals  who  can  challenge  the 
methodology  that  is  employed.  Reviews  of 
those  CEA/CBAs  that  are  not  too  technical, 
however,  may  facilitate  public  scrutiny  regard- 


ing the  validity  and,  especially,  the  appropriate- 
ness of  key  assumptions.  Such  scrutiny  may  be 
useful  because  the  application  of  CEA/CBA  in 
the  field  of  health  policy  is  only  part  of  a  larger 
political  process. 

Since  this  report  is  primarily  designed  to  ex- 
amine the  policy  implications  of  using  CEA/ 
CBA  for  health  care  resource  allocation  deci- 
sions, the  methodological  process  which  is  envi- 
sioned is  substantially  different  from  what 
would  be  discussed  if  this  report  were  being 
written  for  the  academic  research  community. 
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It  is  necessary  to  make  this  distinction  be- 
cause CEA/CBA  can  be  a  very  complex  under- 
taking analytically  and  often  requires  a  massive 
data-gathering  effort.  For  instance,  disease  pro- 
gression rates  must  often  be  assigned  and  math- 
ematical models  must  capture  the  dynamics  of 
the  process;  the  effects  of  medical  intervention 
may  need  to  be  estimated  by  professional  opin- 
ion or  empirically  evaluated  through  epidemio- 
logical observation  or  by  formal  clinical  trials; 
joint  production  costs  may  need  to  be  estimated 
using  sophisticated  dynamic  programing  tech- 
niques; and  so  forth.  All  this  is  expensive,  time 
consuming,  and  is  apt  to  require  very  special- 
ized computer  support,  analytical  skills,  and 
clinical  judgment.  On  the  other  hand,  the  real 
world  dictates  that  health  resource  allocation 
decisions  must  often  be  made  without  the  bene- 
fit of  such  resources — that  is,  with  little  time, 
money,  and  technical  expertise.  These  subopti- 
mal  conditions,  however,  do  not  relieve  deci- 
sionmakers from  the  responsibility  of  weighing 
the  consequences  of  decisions. 

Since  CEA/CBA  is  being  spoken  of  or  advo- 
cated as  a  mechanism  to  assist  policymakers  in 
making  rational  choices  between  competing  ob- 
jectives, OTA  was  asked  to  assess  the  technique 
for  that  purpose.  The  findings  are  that,  as  for- 
mally applied,  this  analytical  method  could 
often  be  too  complex,  expensive,  and  time  con- 
suming if  used  as  a  routine  method  for  decisions 
by  public  policymakers.  In  fact,  the  cost-effec- 
tiveness case  studies  conducted  as  part  of  this 
assessment  serve  to  highlight  the  immaturity  of 
the  technique  itself.3  Initial  drafts  of  more  than 

3See  Background  Paper  #2:  Case  Studies  of  Medical  Technol- 
ogies, Background  Paper  #3:  The  Efficacy  and  Cost  Effectiveness 


half  of  the  studies,  all  of  which  were  performed 
by  respected  health  care  researchers,  were  con- 
sidered by  reviewers  to  be  inadequate  with  re- 
spect to  the  relevancy/usefulness  of  the  results, 
the  validity  of  the  methods,  the  tenuousness  (or 
error)  in  the  key  assumptions,  or  the  validity  of 
the  data  used.  Clearly,  the  field  is  not  yet  fully 
defined. 

Nevertheless,  the  logic  behind  using  CEA/ 
CBA,  even  at  an  operational  or  policymaking 
level,  appears  sufficient  to  suggest  that  the  10 
principles  previously  enumerated  can  and 
should  be  followed  under  most  circumstances. 

In  no  way,  however,  does  this  finding  suggest 
that  a  complete  analysis  is  either  easy  or  unnec- 
essary. There  is  clearly  a  need  for  ongoing  and 
sophisticated  studies  of  the  cost  effectiveness  of 
specific  technologies  as  well  as  a  need  for  ad- 
vancing the  state-of-the-art  itself.  For  instance, 
much  good  research  has  been  done  in  develop- 
ing and  testing  sets  of  indexes  that  describe  the 
health  status  of  a  population  at  any  given  time 
(79).  That  type  of  work  should  continue  and 
perhaps  should  receive  more  emphasis.  Never- 
theless, formal  CEA/CBAs,  however  poten- 
tially valid  and  effective,  can  be  inappropriately 
used  by  decisionmakers  who  lack  the  necessary 
resources  and  skills.  Defining  a  more  practical, 
limited  approach  to  the  methods  seems  clearly 
appropriate  and  does  not  diminish  the  worth  of 
or  need  for  more  sophisticated  approaches  un- 
der different  circumstances. 


of  Psychotherapy,  and  Background  Paper  #5:  Assessment  of  Four 
Common  X-Ray  Procedures,  prepared  by  OTA  in  conjunction 
with  this  assessment. 


NONAGGREGATED  ANALYSIS— AN  ARRAYING  TECHNIQUE 


Since  many  of  the  methodological  weaknesses 
of  CEA/CBA  may  be  hidden,  aggravated,  or  in 
fact  caused  by  the  practice  of  deriving  a  cost- 
benefit  or  cost-effectiveness  ratio — that  is,  a 
numerical  bottom-line — the  possibility  of  not 
aggregating  the  often  complex  sets  of  calcula- 
tions should  be  investigated  and  considered.  In- 
stead of  aggregating,  analyses  might  be  done  by 


explicitly  listing  or  ARRAYING  all  the  elements 
that  are  included  in,  or  would  be  affected  by, 
decisions. 

When  costs  and  effects  can  be  quantified,  that 
would  be  done;  when  they  can  be  combined, 
that  would  also  be  done.  Whenever  one  or  more 
important    nonquantifiable    variables  would 
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otherwise  either  be  left  out  or  be  relegated  to  a 
footnote,  however,  no  effort  to  arrive  at  a  single 
combined  benefit  value  would  be  made. 

A  nonaggregated  or  array  method  of  analysis 
would  give  decisionmakers  a  greater  number  of 
elements  to  consider,  but  it  would  also  make  in- 
tangible or  nonquantifiable  factors  more  expli- 
cit, and  thus  might  help  force  consideration  of 


these  factors  by  decisionmakers  commensurate 
with  the  factors'  significance. 

A  more  detailed  examination  of  this  arraying 
possibility, along  with  a  discussion  of  circum- 
stances leading  to  OTA's  suggesting  it,  is  found 
in  Background  Paper  #2;  Methodological  Issues 
and  Literature  Review. 
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Uses  and  Usefulness  of 
CEA/CBA:  General  Findings 


INTRODUCTION 

Substantial  disagreement  and  confusion  sur- 
round the  question  of  the  potential  usefulness 
of  cost-effectiveness  analysis/cost-benefit  anal- 
ysis (CEA/CBA)  in  decisions  regarding  medical 
technology  and  the  health  care  system.  With  the 
continuing  concern  over  health  care  expendi- 
tures, and  with  the  advocacy  of  CEA/CBA  by 
many  people  and  groups  as  a  means  to  amelio- 
rate cost-related  problems,  this  confusion  and 
disagreement  take  on  a  significance  that  is  far 
more  than  academic. 

OTA  believes  that  the  potential  usefulness  of 
CEA/CBA  depends  very  critically  not  only  on 
the  feasibility  but  also  on  the  implications  of  its 
use.  Accordingly,  in  this  assessment,  OTA  ex- 
amined three  major  issues:  1)  the  general  use- 
fulness (past  and  potential)  of  CEA/CBA  in  de- 
cisionmaking regarding  medical  technology; 
2)  the  methodological  strengths  and  limitations 
of  CEA/CBA;  and  3)  the  potential  for  initiating 
or  expanding  the  use  of  CEA/CBA  in  health 
care  decisionmaking  regarding  medical  tech- 
nology, especially  in  six  health  care  programs — 
reimbursement  coverage,  health  planning,  mar- 
ket approval  for  drugs  and  devices,  Profession- 
al Standards  Review  Organizations  (PSROs), 
R&D  activities,  and  health  maintenance  organi- 
zations (HMOs).  A  major  aspect  of  the  exami- 
nation of  the  third  issue  is  the  potential  impli- 
cations— to  the  programs,  to  decisionmaking 
quality,  and  to  society's  values — of  CEA/CBA 
use  in  the  six  program  areas. 

The  primary  focus  of  this  assessment  is  on  the 
application  of  CEA/CBA  to  medical  technolo- 
gy— which  OTA  defines  as  the  drugs,  devices, 


and  medical  and  surgical  procedures  used  in 
medical  care,  and  the  organizational  and  sup- 
port systems  within  which  such  care  is  pro- 
vided. With  the  exception  of  a  background  pa- 
per on  CEA/CBA  and  psychotherapy,1  the  as- 
sessment does  not  directly  address  psychosocial 
medicine.  Other  aspects  of  health,  such  as  the 
environment,  are  also  not  covered.  OTA  be- 
lieves, however,  and  it  was  the  consensus  of  the 
advisory  panel,  that  the  findings  presented  in 
this  report,  and  in  the  background  paper  on 
methodological  issues,2  may  apply  also  to  other 
areas  such  as  health  care  resource  allocation  in 
general.  With  modification,  the  findings  may 
also  apply  to  areas  such  as  environmental  health 
regulation,  occupational  safety  and  health,  and 
education  resource  allocation. 

Furthermore,  although  the  subject  of  the  as- 
sessment was  CEA/CBA,  the  findings  should 
also  be  examined  with  an  eye  to  their  applicabil- 
ity to  other  types  of  formal  analysis.  Risk-ben- 
efit analysis,  decision  analysis,  systems  anal- 
ysis, technology  assessment,  and  social  impact 
assessment,  for  example,  are  all  techniques  used 
to  examine  various  policy  questions  in  both 
public  and  private  organizations.  The  usefulness 
and  implications  of  each  of  these  techniques  will 
vary  according  to  many  of  the  same  factors  that 
affect  the  usefulness  of  CEA/CBA.  In  fact,  there 
are  only  hazy  distinctions  between  these  other 
forms  of  analysis  and  the  forms  of  CEA/CBA. 


background  Paper  #3;  The  Efficacy  and  Cost  Effectiveness  of 
Psychotherapy,  prepared  by  OTA. 

'Background  Paper  #1:  Methodological  Issues  and  Literature 
Review,  prepared  by  OTA. 
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GENERAL  FINDINGS 

OTA  found  few  examples  of  well-conducted, 
sophisticated  CEA/CBAs  conducted  for  and 
used  in  decisionmaking  in  health  care.  It  is  like- 
ly, however,  that  the  extent  of  use  of  CEA/ 
CBAs  in  health  care  decisionmaking  OTA 
found  in  its  survey  (see  app.  B)  understates  ac- 
tual usage — of  informal  CEAs  in  particular,  but 
of  formal,  relatively  sophisticated  analyses,  as 
well.  OTA's  survey  was  not  exhaustive.  The  ef- 
fort that  was  undertaken  to  ascertain  the 
amount  of  use,  though,  does  seem  to  indicate 
that  the  level  of  use  is  not  substantial.  Use  of 
formal  CEA/CBA  in  decisionmaking  in  health 
care  is  the  exception  not  the  rule. 

It  is  safe  to  say,  however,  that  most  decisions 
made  take  into  account  only  a  subset  of  the  po- 
tential consequences  of  those  decisions.  The  in- 
herent complexities  of  many  decisions  and  the 
uncertainties  of  decision  variables  make  it  ex- 
tremely difficult  to  identify  and  weigh  all  the 
consequences.  In  general,  OTA  found,  the  qual- 
ity and  validity  of  decisions  can  be  increased  by 
analysis  that  forces  a  structuring  of  the  decision 
process,  that  provides  a  framework  for  identify- 
ing and  considering  as  many  of  the  relevant 
costs  and  benefits  as  is  feasible. 

This  finding  supports  the  two  major  general 
findings  of  the  assessment  that  were  presented  at 
the  end  of  chapter  2.  The  process  of  CEA/CBA 
may  be  more  helpful  generally  than  would  be 
the  rigid  and  formal  application  of  CEA/CBA 
study  results  in  health  care  programs. 

Chapter  2  also  set  out  two  broad  classes 
of  health  care  program  decisions:  constrained 
budget  ones,  and  nonbudget  or  nonconstrained 
ones.  CEA/CBA  potentially  can  be  more  valua- 
ble for  decisionmaking  under  a  constrained 
budget  where  tradeoffs  have  to  be  made  directly 
than  when  constraints  are  nonexistent  or  very 
indirect.  Under  the  budget  system,  the  budget  it- 
self would  act  as  a  cost  containing  or  controlling 
factor.  Under  the  nonconstrained  type  of  sys- 
tem, since  no  direct  tradeoffs  are  required,  no 
direct  limit  on  expenditures  is  set  or  forced. 
Thus,  in  neither  case  would  CEA/CBA  neces- 
sarily function  as  an  effective  cost-constraining 
mechanism  or  tool.  Advocacy  of  CEA/CBA  as 


such  a  tool,  therefore,  should  be  regarded  skep- 
tically. CEA/CBA  might,  though,  change  the 
mix  of  expenditures.  Technologies  might  be  sub- 
stituted for  one  another  on  the  basis  or  partially 
on  the  basis  of  analysis — especially  under  a 
budget  situation.  In  this  regard,  there  is  poten- 
tial for  CEA/CBA  to  help  increase  efficiency, 
even  in  terms  of  health  outcome,  without  neces- 
sarily lowering  total  expenditures. 

Related  to  this  last  point  about  the  possible 
use  of  CEA/CBA  in  improving  resource  alloca- 
tion within  a  constrained  budget  is  the  observa- 
tion that  this  country's  health  care  system  might 
move  closer  to  an  overall  constrained  budget. 
This  is  mentioned  only  as  a  possibility.  Perhaps 
in  the  future,  health  care  expenditures  may  be 
limited  (or  constrained)  to  a  fixed  or  specified 
percentage  of  gross  national  product,  or  to  some 
specified  absolute  amount  of  dollars.  If  this  situ- 
ation ever  comes  about,  or  even  as  an  increasing 
number  of  individual  institutions  and  programs 
operate  under  budget  constraints,  the  appropri- 
ateness of  CEA/CBA  may  increase.  In  such  a 
possible  future  situation,  most  resource  alloca- 
tion decisions  would  require  explicit  tradeoffs. 
It  is  conceivable,  therefore,  that  efforts  devoted 
to  the  development  of  a  CEA/CBA-based  ap- 
proach to  decisionmaking  (not  necessarily  tied 
to  numerical  study  results)  will  represent  an 
investment  in  future  social  policymaking.  The 
lack  of  direct  applicability  of  formal  CEA/CBA 
to  many  of  today's  decisions  may  then  be  offset 
by  future  applications  of  CEA/CBA. 

Various  sources  consulted  and  numerous  peo- 
ple interviewed  by  OTA  for  this  assessment  pro- 
vided information  yielding  several  seeming  con- 
tradictions or  paradoxes  concerning  CEA/CBA. 
For  example,  one  common  argument  is  that  use 
of  CEA/CBA  may  often  be  unethical  if  it  does 
not  take  values  and  distributional  issues  into  ac- 
count adequately.  There  is  validity  to  that  state- 
ment. But  there  also  seems  to  be  validity  to  the 
argument  that  not  considering  costs  and  benefits 
in  decisions  on  society's  resources,  especially  in 
an  area  so  basic  as  health,  is  unethical,  because 
in  the  absence  of  the  explicit  consideration  of 
consequences  and  of  the  parties  on  whom  those 
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consequences  may  fall,  inequities  will  very  like- 
ly occur. 

Another  example  of  a  seeming  paradox  con- 
cerns the  "power"  of  CEA/CBA  results.  Some 
people  argued  that  because  many  decisions  are 
made  in  a  political  context,  the  results  of  any 
"objective"  analysis  would  be  heavily  criticized 
and  overwhelmed  by  other  factors.  Yet  others 
argue  that  one  of  the  factors  in  the  potential 
misuse  of  CEA/CBA  is  its  quantitative  nature, 
allowing  those  involved  in  the  decision  process 
to  "anchor"  their  arguments  to  what  appear  to 
be  hard  numbers.  Are  the  results  of  CEA/CBA 
powerless?  Or  overly  powerful? 

The  resolution  of  both  these  examples  may  lie 
in  the  distinction  between  the  process  or  ap- 
proach of  CEA/CBA  and  the  quantitative  re- 
sults of  formal  studies.  As  indicated  by  the  two 
general  findings  of  this  assessment,  many  of  the 
negative  perceptions  of  CEA/CBA  are  based  on 
the  possible  misuse  or  inappropriate  use  of  for- 
mal study  results.  Viewed  as  a  method  of  struc- 
turing the  decision  process,  CEA/CBA  need  not 
hide  or  avoid  questions  of  ethics  or  values,  and 
it  need  not  provide  a  deceptively  quantitative 
answer  to  complex  problems. 

As  an  example  of  the  difficulty  of  concentrat- 
ing on  quantifiable  variables  and  how  investiga- 
tions of  decision  possibilities  might  be  enhanced 


by  thinking  in  CEA/CBA  terms,  consider  the 
cost  effectiveness  of  CEA/CBA  itself.  OTA  was 
frequently  asked  whether  a  CEA/CBA  of  CEA/ 
CBA  might  not  be  what  is  needed.  And  for  a 
given  decision  situation  that  type  of  analysis 
might  be  very  valuable.  Approaching  a  CEA/ 
CBA  of  CEA/CBA  in  order  to  arrive  at  a  quan- 
titative, traditional  bottom-line  result,  how- 
ever, might  lead  analysts  to  list  as  a  primary 
"cost"  of  CEA/ CBA  the  resource  costs  involved 
in  conducting  and  interpreting  the  studies. 
Thus,  resource  costs  such  as  those  identified  in 
appendix  C  would  be  included,  with  dollars  be- 
ing the  measure  used.  If,  however,  the  analysts 
were  less  interested  in  a  bottom-line  figure  for 
the  CEA/CBA  of  CEA/CBA,  they  might  con- 
sider the  opportunity  costs  of  analyses.  That  is, 
the  more  important  aspect  of  the  costs  of  CEA/ 
CBA  may  not  be  the  dollars  it  takes  to  conduct 
it,  but  rather  the  alternate  uses  of  those  dollars 
and  the  alternate  types  of  analysis  and  other  ac- 
tivities that  might  occupy  the  attention  of  those 
concerned  about  more  rational  allocation  of 
medical  technologies  (617).  Would  the  funds 
and  attention  that  could  be  devoted  to  CEA/ 
CBA  be  more  productive  if  applied  to  efficacy 
and  safety  studies?  To  education  or  conscious- 
ness-raising of  physicians?  To  more  dissemina- 
tion of  existing  knowledge  of  the  costs  and  bene- 
fits of  various  technologies?  To  regulation  of  the 
use  of  technology?  These  are  the  questions  that 
probably  should  be  asked. 


FACTORS  AFFECTING  THE  USE  AND  USEFULNESS  OF  CEA/CBA 


One  of  the  key  factors  affecting  the  uses  and 
usefulness  of  CEA/CBA  has  already  been  dis- 
cussed in  chapter  3:  the  technical,  methodologi- 
cal feasibility  of  the  technique.  These  methodo- 
logical factors  can  be  inherent  aspects  of  CEA/ 
CBA,  or  they  can  be  due  to  the  state-of-the-art 
of  CEA/CBA  and  thus  more  tractable. 

The  manner  in  which  both  types  affect  the 
usefulness  of  CEA/CBA,  however,  should  be 
analyzed  in  the  context  or  the  environment  of 
current  or  potential  uses  of  CEA/CBA.  In  other 
words,  the  questions  should  be  asked  what  is  the 
decisionmaking  context  and  how  does  it  affect 


the  strengths  or  limitations  of  the  methodology, 
and  vice  versa?  For  example,  does  the  decision 
relate  to  a  technology  at  an  early  stage  in  its  life- 
cycle,  such  as  bone  marrow  transplants?  Or 
does  it  concern  an  established  technology,  such 
as  appendectomy?  Is  the  technology  in  question 
a  diagnostic  technology,  such  as  the  CT  scan- 
ner, or  a  therapeutic  one,  such  as  renal  dialysis? 

The  possibility  of  affecting  the  course  of  a 
technology's  diffusion  and  use  might  be  greater 
in  early  stages  of  its  development,  but  the  uncer- 
tainties about  its  health  effects  and  its  costs  will 
generally  be  greater.  Thus,  it  may  be  possible  to 
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do  a  more  valid  or  certain  CEA/CBA  later  in 
the  technology's  lifecycle,  but  the  information 
gained  may  be  less  valuable  for  public  policy. 
The  tradeoff  required  will  vary  depending  on 
the  specifics  of  the  technology  and  the  policy  de- 
cision to  be  made.  In  addition,  diagnostic  tech- 
nologies are  often  more  difficult  to  study  than 
other  technologies  because  of  the  uncertainties 
involved  in  linking  their  use  to  health  outcomes. 
Thus,  studies  of  diagnostic  technologies  often 
tend  toward  the  "net  cost"  end  of  the  CEA/CBA 
spectrum,  where  the  measures  of  outcome  or 
benefit  may  be  numbers  of  tests  performed  or 
levels  of  diagnostic  accuracy. 

In  sum,  the  stage  of  development  of  the  tech- 
nology under  study  and  the  type  of  technology 
(or  function  of  the  technology)  are  two  of  the 
factors  that  will  affect  the  specifics  of  analysis  to 
be  used,  the  uses  to  which  analysis  can  be  put, 
and  the  usefulness  of  resultant  information. 
Other  factors  are  the  relative  strength  or  im- 
portance of  nonanalytical  factors,  such  as  poli- 
tics or  equity,  in  the  decisions  to  be  made;  the 
ability  of  the  sponsors  of  analysis  to  implement 
the  results;  the  familiarity  of  sponsors  and  deci- 
sionmakers with  formal  analysis;  the  existence 
of  adequate  data  relating  to  the  technology,  to 
the  disease  or  other  problem  addressed  by  the 
technology,  or  on  other  possible  effects  of  inter- 
ventions based  on  analysis;  the  existence  of  eco- 
nomic incentives  that  match  or  run  counter  to 
the  results  of  analysis,  the  types  of  decisions  to 
be  made  (e.g.,  budget-based  decisions  or  non- 
budget-based  decisions);  and  so  on.  Some  of  the 
factors  that  affect  the  use  of  CEA/CBA  are 
listed  in  table  2. 

One  of  the  factors  listed  above  is  of  particular 
importance:  the  quality  and  availability  of  data. 
Obviously,  without  data  or  estimates  of  data, 
there  would  be  no  CEA/CBAs.  The  quality  of  a 
CEA/CBA  is  directly  related  to  the  accuracy  of 
the  data  used  in  it.  For  example,  when  good  epi- 
demiological data  on  the  effects  of  a  technology 
or  the  existence  of  disease  are  present,  analysis 
will  have  a  greater  potential  for  being  relevant 
and  useful.  A  specific  example  of  where  epi- 
demiological data  have  permitted  analyses  of 
high  quality  is  in  the  area  of  smoking  and  its  ef- 
fects on  health.  Good  data  do  not  guarantee 
good  analyses,  however,  because  the  quality  of 


Table  2.— Factors  Affecting  the  Use  of  CEA/CBA 

Stage  of  Development  of  the  Technologies  Under  Study. 

—Tradeoff  required  between  availability/validity  of  data 
and  ability  to  affect  the  future  use  of  the  technologies. 
Both  the  type  of  analysis  and  the  usefulness  of  analysis 
will  be  affected. 

Nature  of  Technologies  Under  Study  and  Function  of  Tech- 
nologies Under  Study.— In  terms  of  function,  diagnostic 
technologies,  for  example,  often  have  indirect  connec- 
tions to  health  outcome  and  often  lend  themselves  to  the 
net  cost  type  of  CEA/CBA.  In  terms  of  the  physical  nature 
of  technologies,  surgery,  for  example,  may  involve  addi- 
tional uncertainties  due  to  varying  skills  of  surgeons  and 
surgical  settings.  Both  type  and  use  of  analysis  will  be  af- 
fected, but  especially  the  type  or  specific  methodological 
elements. 

,  Social,  Ethical,  or  Value  Influences  in  the  Decision  Environ- 
ment.—Very  similar,  often  overlapping  with  the  above 
factor.  Will  affect  both  the  type  and  uses  of  the  analysis. 
The  example  of  renal  dialysis  applies  here.  Abortion 
would  serve  as  another  example. 

Quality  of  the  Analysis.— Can  be  of  at  least  four  types: 
Analysis  Subject  to  Inherent  Methodological  Limitations. 
—e.g.,  inability  to  adequately  deal  with  equity  con- 
cerns; influence  of  discount  rate  chosen  on  outcome 
of  analysis. 

Analysis  Subject  to  State-of-the-Art  Limitations.— e.g., 
difficulties  in  identifying  and  measuring  many  costs  or 
effects. 

Analysis  Containing  Errors  of  Omission  or  Commission. 
—These  are  errors  not  due  to  the  state-of-the-art,  e.g., 
failure  to  discount  or  perform  sensitivity  analysis  when 
appropriate. 

Analysis  Subject  to  Data  Limitations.— This  factor  will  af- 
fect quality  even  though  the  other  factors  might  have 
been  adequately  dealt  with.  Much  cost  and  health  out- 
come data  are  uncertain,  difficult  to  retrieve,  or  simply 
nonexistent. 

All  four  of  these  factors  can  affect  the  quality  of  analysis, 
which  in  turn  affects  the  usefulness  of  the  results. 

Ability  of  Sponsors  or  Users  of  Analysis  to  Implement  Re- 
sults.—The  usefulness  of  analysis  will  naturally  depend 
on  the  amount  of  control  the  user  has  over  the  particular 
technology  or  situation  studied. 

Experience/Familiarity  of  Users  With  the  Type  of  Analysis 
Conducted.— This  factor  will  affect  usefulness  in  two 
ways:  it  will  be  a  direct  influence  on  the  acceptability  of 
results,  and  it  will  affect  the  ability  of  the  users  to  appro- 
priately apply  the  results. 

Existence  of  Economic  Incentives  in  the  Decision  Environ- 
ment.—If  the  economic  incentives  relating  to  the  use  of 
the  technology  under  study  are  in  accord  with  the  results, 
their  acceptability  will  be  great.  If  they  run  counter  to  the 
results,  the  usefulness  will  be  limited,  depending  on  the 
strength  of  the  economic  incentives. 

SOURCE:  Office  of  Technology  Assessment. 

analysis  is  also  affected  by  the  other  factors 
mentioned  above.  Similarly,  the  usefulness  of 
analysis  is  dependent  on  those  factors  affecting 
quality  as  well  as  on  a  number  of  other  factors 
(see  table  2)  relating  to  the  decisionmaking  and 
analytical  contexts  or  environments. 
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There  are  many  gaps  in  the  data  available  for 
CEA/CBA,  owing  to  such  factors  as  methodo- 
logical constraints,  inadequate  resources  for 
data  collection  and  interpretation,  lack  of  com- 
munication between  the  users  of  data  and  those 
collecting  it,  and  the  sheer  impossibility  of  col- 
lecting and  analyzing  all  the  data  that  could  be 
used  by  someone,  somewhere.  The  principal 


Federal  agency  charged  with  collecting  and  ana- 
lyzing health  data  is  the  National  Center  for 
Health  Statistics  (NCHS).  NCHS  is  currently  in- 
volved in  several  developmental  projects  in- 
tended to  clarify  certain  methodological  issues 
related  to  the  provision  of  data  for  CEA/CBA, 
especially  in  relation  to  cost-of-illness  studies 
(see  app.  B). 


POTENTIAL  USERS  OF  CEA/CBA 

Health  care  policies  and  other  decisions  are 
made  at  a  variety  of  levels  and  in  a  variety  of 
situations  by  an  extremely  broad  range  of  indi- 
viduals and  groups.  In  theory,  CEA/CBA  re- 
sults or  approaches  might  be  useful  to  any  or  all 
of  these  decisionmakers.  Table  3  lists  many  of 
the  decisionmakers — the  list  is  not  exhaustive 
but  should  provide  an  idea  of  how  diverse  and 
numerous  the  types  of  decisionmakers  are. 
Three  general  classes  of  decisionmakers  or  po- 
tential users  of  CEA/CBA  information  are  dis- 
cussed in  this  assessment:  individual  medical 

Table  3.— Partial  List  of  Individuals  and  Groups 
Making  or  Influencing  Resource  Allocation 
Decisions 


Individual  physicians  and  other  health  care  professionals 
Individual  patients 

Medical  professional  societies  and  boards 
Consumer  groups 

Health  industry  representatives  and  organizations 
Hospitals,  clinics,  other  health  care  institutions 
Labor  organizations 
Businesses 

Health  maintenance  organizations 

Medicare  and  medicaid 

Other  governmental  health  care  programs 

Health  systems  agencies,  State  agencies 

Professional  Standards  Review  Organizations 

Blue  Cross  and  Blue  Shield  Associations 

Other  health  care  insurers,  third-party  payers 

Other  quality  assurance  or  utilization  review  groups 

Food  and  Drug  Administration 

Ratesetting  commissions 

Voluntary  health  organizations 

Public  health  departments 

Other  State  and  local  health  agencies 

U.S.  Congress,  executive  agencies,  State  legislatures 

Health  care  systems,  such  as  the  Veterans  Administration's 

and  the  Department  of  Defense's 
Medical  schools 

Biomedical  and  health  services  researchers 
Other  health-related  associations 


SOURCE:  Office  of  Technology  Assessment. 


practitioners,  nongovernmental  institutions, 
and  governmental /quasi-governmental  institu- 
tions. 

Individual  Medical  Practitioners 

Despite  the  fairly  small  amount  of  empirical 
research  on  the  subject,  it  seems  safe  to  say  that 
CEA/CBA  has  had  little  direct  impact  on  indi- 
vidual physicians'  behavior.3 

Discussions  with  academic  physicians  indi- 
cate a  consensus  regarding  the  above  point  on 
CEA/CBA's  lack  of  impact.  Beyond  that  point, 
however,  the  consensus  dissolves.  There  is  dis- 
agreement, for  example,  concerning  whether 
CEA/CBA  has,  and  if  so  the  extent  to  which, 
significantly  affected  physicians'  consciousness 
of  economic  issues.  Explanations  for  the  lack  of 
impact  on  practice  are  numerous,  with  emphasis 
on  their  relative  importance  varying  dramatical- 
ly from  one  observer  to  the  next.  And  the  con- 
sensus on  current  practice  impact  does  not 
translate  into  agreement  on  the  future  role  of 
CEA/CBA  in  influencing  individual  physician 
behavior:  Opinion  seems  to  be  split  roughly  in 
half  between  those  who  believe  that  CEA/CBA 
will  cause  many  physicians  to  alter  their  medical 
practices  and  those  who  anticipate  continuation 
of  the  current  absence  of  significant  effect. 

The  principal  explanations  for  CEA/CBA's 
lack  of  impact  on  physicians'  behavior  to  date 
can  be  grouped  under  two  headings: 

1.  The  novelty  of  CEA/CBA  in  health  care. 

Until  very  recently,  the  literature  on  health  care 


3The  following  discussion  is  taken  from  work  done  for  OTA  by 
Kenneth  Warner  of  the  University  of  Michigan  (615). 
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CEA/CBA  was  sparse.  As  indicated  in  Back- 
ground Paper  #2,  this  has  been  particularly  true 
in  the  medical  literature.  Relatively  few  physi- 
cians read  the  nonmedical  health  care  literature; 
hence  their  exposure  to  the  concepts  and  prac- 
tice of  CEA/CBA  was  minimal  prior  to  the  last 
few  years.  Needless  to  say,  lack  of  exposure  cor- 
related highly  with  (and  presumably  caused  in 
part)  a  lack  of  understanding  of  the  technique 
and  meaning  of  CEA/CBA. 

The  novelty  of  CEA/CBA  in  health  care  ac- 
counts for  some  of  the  quality  problems  in  the 
published  literature.  While  poor  analytical  qual- 
ity certainly  could  be  a  barrier  to  application  of  * 
the  results  of  analysis,  few  observers  cite  it  as  a 
significant  factor  in  the  failure  of  physicians  to 
apply  findings  to  their  practices. 

In  a  similar  vein,  the  uncertainties  in  analysis 
frequently  prevent  determination  of  an  unequiv- 
ocal conclusion  in  an  analysis.  Even  when  a  firm 
"bottom  line"  is  presented,  nonquantified  fac- 
tors— for  example,  the  distribution  of  costs  and 
benefits — can  make  the  conclusion  far  from  de- 
finitive. Thus,  one  could  argue  that  even  high- 
quality  analyses  frequently  do  not  produce  find- 
ings that  can  or  should  be  translated  directly  in- 
to practice  by  individual  physician  decision- 
makers. This  seems  an  attractive  explanation 
for  physicians'  nonresponse  to  analysis,  partic- 
ularly combined  with  whatever  bewilderment 
they  may  feel  as  a  result  of  their  unfamiliarity 
with  CEA/CBA.  It  is  not,  however,  an  explana- 
tion often  noted  in  discussions  on  the  subject. 
Most  likely,  this  explanation  presupposes  that 
other,  preliminary  barriers  to  application  of 
analysis  have  been  surmounted;  the  evidence  is 
to  the  contrary.  Thus,  one  might  anticipate  that 
such  inherent  technical  limitations  of  analysis 
will  grow  in  importance  as  other  barriers  fall. 

2.  The  irrelevance  of  much  of  CEA/CBA  to 
medical  practice  decisionmaking.  There  are  two 
basic  sources  of  irrelevance,  one  substantive, 
one  structural.  In  the  substantive  category, 
many  CEA/CBAs  have  involved  assessments  of 
the  desirability  of  social  programs  where  social, 
and  not  individual,  decisionmaking  was  at 
issue.  Examples  include  the  several  studies  of 
communicable  disease  control  programs  (e.g., 
measles,  swine  flu,  etc.),  community  (or  indus- 


try) based  screening  programs  (e.g.,  hyperten- 
sion), and  fluoridation  of  municipal  water  sup- 
plies. The  subject  matter  of  such  studies  pre- 
cludes a  direct  practice  response  by  individual 
physicians. 

While  this  too  may  serve  as  a  useful  partial 
explanation  of  the  absence  of  behavioral  re- 
sponse by  individual  practitioners,  it  cannot  ex- 
plain the  total  absence  of  such  response,  since 
much  of  the  health  care  CEA/CBA  literature  is 
clearly  relevant  to  individual  practice  decision- 
making. Nor  is  this  an  often-cited  explanation. 
A  more  cogent  argument  concerns  structural  ir- 
relevance: According  to  a  strict  economic  inter- 
pretation, most  physicians'  interests  in  "cost- 
effective  care"  deviate  significantly  from  those 
of  society.  All  physicians  share  an  interest  in 
understanding  the  efficacy  and  safety  of  medical 
technologies — technologies  whose  risks  out- 
weigh medical  benefits  are  undesirable — but 
concerns  with  the  economic  side  of  cost  effec- 
tiveness are  either  nonexistent  or  dependent  on 
the  physicians'  economic  environment.  In  gen- 
eral (619): 

Cost  data  are  psychologically  remote.  (The 
physician's)  one-on-one  relationship  with  the 
patient  is  not  in  the  context  of  the  cost  to  socie- 
ty. 

The  physician's  economic  circumstances, 
however,  can  produce  in  the  physician  an  often 
subconscious  reaction  to  costs.  To  a  fee-for- 
service  physician  whose  patients  are  well  in- 
sured, the  cost  of  a  technology  may  be  irrele- 
vant, at  least  immediately.  If  the  physician 
works  within  the  context  of  prepayment,  how- 
ever, the  professional  concern  with  cost  effec- 
tiveness begins  to  approach  the  social  concern. 
In  all  cases,  the  patient's  economic  wherewithall 
often  will  be  a  major  consideration:  In  an  en- 
vironment of  prepayment  or  adequate  insurance 
coverage,  high  costs  of  technologies  do  not 
translate  into  direct  economic  burdens  on  pa- 
tients; hence  the  high  costs  are  something  of  an 
abstraction  to  both  the  immediate  patient  and 
the  physician. 

This  economic  interpretation — emphasized 
by  many  knowledgeable  observers — attributes 
the  lack  of  effect  of  CEA/CBA  on  medical  prac- 
tice to  its  irrelevancy  and  even  inconsistency 
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with  medical  norms,  irrespective  of  the  quality 
or  quantity  of  the  literature.  Accordingly,  un- 
less the  reimbursement  system  is  changed,  this 
argument  suggests,  the  future  will  auger  little 
change  in  the  application  of  CEA/CBA  to  indi- 
vidual practice  decisionmaking.  According  to 
this  explanation,  physicians'  nonresponse  to 
CEA/CBA  is  not  necessarily  a  reflection  of 
physicians'  selfish  monetary  interests,  or  their 
indifference  to  economic  considerations.  Ra- 
ther, nonresponse  to  CEA/CBA  perhaps  reflects 
physicians'  fulfilling  their  roles  as  agents  of  their 
clients — patients.  A  physician's  major  responsi- 
bility may  be  to  weigh  all  the  costs  and  benefits 
to  the  patient  and  to  his  or  her  medical  prac- 
tice— i.e.,  the  aggregate  of  all  the  patients  of  the 
physician. 

This  argument  is  not  an  entirely  economic 
one,  because  the  ethics  of  the  doctor-patient  re- 
lationship are  involved.  If  a  patient  is  not 
harmed  economically  by  performance  of  a  cer- 
tain procedure,  even  though  only  a  small  medi- 
cal benefit  might  be  expected,  what  are  the 
ethics  of  the  individual  physician's  denying  or 
recommending  against  the  procedure  in  order  to 
represent  society's  cost  and  benefit  priorities? 
The  differences  between  social  and  individual 
economic  and  ethical  considerations  constitute 
the  only  frequently  advanced  explanation  for 
physicians'  nonresponse  to  CEA/CBA  that  does 
not  imply  a  brighter  future  for  the  ability  of 
analysis  to  alter  individuals'  medical  practice 
policies.  Systemwide  changes  in  the  economic 
environment,  such  as  growth  in  HMOs  or  major 
reimbursement  reforms,  might  more  closely 
aline  the  practice  of  medicine  with  the  precepts 
of  analysis.  The  strength  of  the  explanation  does 
not  depend  on  lack  of  understanding  of  CEA/ 
CBA  within  the  medical  community;  hence  an- 
ticipated increases  in  familiarity  with  analysis 
need  not  promote  the  direct  application  of  find- 
ings. Accordingly,  barring  external  pressures, 
the  economic  incentives  and  ethical  norms  of 
medicine  may  very  well  continue  to  preclude 
widespread  application  by  practitioners  of  the 
findings  of  health  care  CEA/CBAs,  with  the  ex- 
ception of  the  "easy"  cases  in  which  one  proce- 
dure is  demonstrated  to  be  both  more  effective 
and  less  costly  than  an  alternative. 


Nongovernmental  Institutions 

A  variety  of  nongovernmental  institutions  are 
potential  consumers  of  CEA/CBAs.  Insurers 
have  a  direct  economic  incentive  to  find  and 
promote  cost  effectiveness  in  the  provision  of 
health  care  services;  officials  of  major  insurers, 
including  Blue  Cross/Blue  Shield,  have  ex- 
pressed their  interest  in  the  development  of 
more  and  better  CEA/CBAs  to  assist  them  with 
reimbursement  decisions  (see  ch.  5).  In  an  era  of 
increasing  restrictions  on  reimbursement,  hos- 
pitals' interests  in  enhancing  efficiency  are  ob- 
vious. HMOs  also  have  a  direct  economic  inter- 
est in  cost-effective  care:  Greater  efficiency 
translates  into  lower,  more  competitive  mem- 
bership rates  and/ or  higher  incomes  for  member 
physicians.  Large  business  firms  and  unions 
have  several  reasons  to  be  interested  in  CEA/ 
CBA:  Greater  efficiency  in  the  provision  of 
medical  services  to  employees  implies  lower 
business  costs  or  room  for  negotiation  of  other 
fringe  benefits;  health  promotion  and  disease 
prevention  among  workers  may  increase  pro- 
ductivity and  reduce  other  costs  of  disability 
and  morbidity;  and  so  on.  As  major  financers  of 
the  costs  of  illness,  each  of  these  organizations 
has  not  only  an  interest  in  promoting  cost-effec- 
tive care,  but  also  has  the  market  power  to 
translate  judgments  concerning  cost  effective- 
ness into  changes  in  health  practice. 

Another  group  of  nongovernmental  institu- 
tions, not  directly  involved  in  the  financing  of 
care,  is  evidencing  interest  in  CEA/CBA:  pro- 
fessional associations.  Among  such  groups  are 
the  Association  of  American  Medical  Colleges, 
the  American  College  of  Physicians,  the  Resi- 
dent Physicians  section  of  the  American  Med- 
ical Association,  and  the  cost-containment  com- 
mittees of  several  State  medical  societies.  In 
part,  the  interest  of  such  groups  reflects  con- 
cerns about  the  social  implications  of  inefficient 
medical  resource  allocation;  in  part,  interest 
may  reflect  a  perception  that  if  the  health  care 
community  does  not  control  cost  inflation, 
Government  regulators  may  attempt  to  do  the 
job  for  it.  Regardless  of  the  motivation,  the 
demonstrated  interest  suggests  a  receptivity  to 
information  that  CEA/CBA  in  theory  can  pro- 
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vide.  This  interest  extends  beyond  the  medical 
community.  The  American  Dietetic  Association 
has  recently  completed  a  study  of  the  costs  and 
benefits  of  nutrition  care  services;  dentists  have 
discussed  the  relative  efficiency  of  alternative 
methods  of  preventing  caries  (78);  and  so  on. 
Whether  the  efforts  of  such  groups  will  ever 
translate  into  significant  practice  changes  re- 
mains to  be  seen.  But  the  "cost  consciousness- 
raising"  function  of  CEA/CBA  seems  well 
served  by  such  efforts. 

To  date,  direct  application  of  CEA/CBAs  to 
institutional  policies  has  been  limited.  Some 
organizations  have  identified  themselves  as  be-, 
ing  in  the  market  for  specific  analyses — for  ex- 
ample, several  businesses  want  to  learn  more 
about  the  costs  and  benefits  of  certain  disease 
prevention/health  promotion  programs  for  em- 
ployees (e.g.,  control  of  alcoholism  and  drug 
abuse,  hypertension  screening  and  treatment, 
executive  exercise  programs,  antismoking  pro- 
grams)— and  analysts  have  responded  with 
CEA/CBAs  tailored  to  the  specific  institutions' 
needs  (275).  Recent  policy  decisions  of  other 
organizations  reflect  a  CEA/CBA  mode  of 
thinking,  though  the  decisions  have  not  derived 
from  formal  CEA/CBAs.  For  example,  the  na- 
tional Blue  Cross  and  Blue  Shield  Associations 
have  recommended  that  member  plans  not  re- 
imburse for  institutionally  standardized  bat- 
teries of  laboratory  tests  on  admission  to  a  hos- 
pital. 

The  question  remains  whether,  and  if  so  how, 
nongovernmental  institutions  will  use  CEA/ 
CBAs.  Certainly,  by  virtue  of  its  ability  to 
clarify  issues  and  collect  and  organize  informa- 
tion, CEA/CBA  could  assist  planning  and  deci- 
sionmaking within  many  of  these  organizations. 
Some  kinds  of  findings  might  lend  themselves 
neatly  to  policy  decisionmaking.  For  example, 
persuasive  evidence  that  a  certain  diagnostic 
procedure  is  both  more  expensive  and  less  ac- 
curate than  an  alternative  procedure  could  serve 
as  solid  grounds  for  nonreimbursement  or  non- 
acquisition  of  the  former.  A  large  cost  differen- 
tial between  two  equally  effective  procedures 
might  also  serve  as  support  for  a  use-constrain- 
ing policy  decision,  though  opposition  might  be 
substantial  if  significant  elements  of  the  medical 


community  questioned  the  procedures'  equality 
of  effectiveness.  Indeed,  whenever  significant 
technical  disagreement  on  efficacy  exists  within 
the  medical  community,  CEA/CBAs  seem  un- 
likely to  overcome  opposition  to  the  policies 
they  might  recommend,  possibly  barring  the 
case  of  a  truly  dramatic  cost  difference. 

This  point  deserves  emphasis  because  of  a 
major  implication:  Clear-cut,  unobjectionable 
CEA/CBA  results  probably  will  be  an  excep- 
tion, not  the  rule.  Furthermore,  they  seem  likely 
to  reflect  reasonably  obvious  differences  be- 
tween the  alternatives  being  studied.  When  a 
CEA/CBA  is  undertaken  out  of  genuine  interest 
in  evaluating  alternatives,  without  significant 
prior  expectations  as  to  the  outcome  of  the  anal- 
ysis, that  outcome  is  less  likely  to  be  definitive. 
Competing  professional  opinions  on  technical 
issues  (e.g.,  diagnostic  accuracy)  exacerbate  the 
problem.  Thus,  definitive  CEA/CBAs  may  sup- 
port policy  decisions,  but  their  potential  to 
shape  such  decisions  seems  limited  by  technical 
and  political  factors. 

Governmental  and 
Quasi-Governmental  Institutions 

A  limitation  of  the  preceding  discussions  is 
that  they  deal  with  the  decisionmakers  as  classes 
of  decisionmakers  (e.g.,  nongovernmental  insti- 
tutions, not  a  specific  institution).  The  discus- 
sion and  arguments  will  vary  according  to  indi- 
vidual circumstances.  For  this  reason,  and  be- 
cause its  mandate  is  related  to  Federal  pro- 
grams, OTA  analyzed  the  potential  use  of  CEA/ 
CBA  by  several  individual  Federal  programs. 

As  an  example  of  a  reimbursement  program, 
medicare  is  used,  but  Blue  Cross  and  Blue  Shield 
are  discussed  briefly  (ch.  5).  The  drug  and 
device  market  approval  processes  of  the  Food 
and  Drug  Administration  are  also  covered  (ch. 
8).  There  are  two  examples  of  programs  that  are 
federally  sponsored,  with  national  policies  and 
administration  to  a  degree,  but  primarily  car- 
ried out  by  quasi-governmental  organizations  at 
the  State  and  local  levels:  the  health  planning 
program  (ch.  7)  and  the  PSROs'  programs  (ch. 
6).  The  usefulness  of  CEA/CBA  to  the  health 
care  R&D  activities  of  the  Federal  Government 
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is  examined,  using  four  Federal  agencies  as  ex- 
amples (ch.  9).  Finally,  a  federally  promoted 
program  that  is  carried  out  by  private  sector 
organizations,  HMOs,  is  discussed  (ch.  10). 

Each  of  these  programs,  or  decision  areas, 
uses  somewhat  different  approaches  to  prob- 
lems and  decisionmaking.  Specific  approaches 
have  been  developed  to  address  divergent  and 
diverse  issues.  Distinct  mechanisms  to  analyze 
decisions  have  been  evolved  in  the  various  pro- 
grams. Although  informal  and  implicit  analysis 
of  costs  and  benefits  seems  to  be  a  frequent 
aspect  of  policy  formulation  in  most  of  the  pro- 
grams, however,  OTA  found  very  little  formal 
use  of  CEA/CBA.  In  several  of  the  areas,  one 
being  market  approval,  cost  itself  has  played  lit- 
tle or  no  role  in  decisions.  Figure  1  provides  a 
view  of  the  relationship  of  the  six  programs  to 
each  other  and  to  the  lifecycle  of  medical  tech- 
nologies. Table  4  is  a  narrative  explanation 
of  that  figure.  An  organization  chart  of  the 
Department  of  Health  and  Human  Services 
(DHHS),  as  seen  from  a  perspective  of  interest 
in  medical  technology,  is  presented  in  figure  2  in 
order  to  show  the  organizations  of  chapters  6 
through  10  in  relation  to  other  elements  of 
DHHS. 

The  Federal  Government  is  not  the  only  pub- 
lic institution  interested  in  analysis.  Given  med- 
icaid and  other  health  care  finance  programs, 
States  share  the  Federal  Government's  concern 
with  health  cost  inflation.  Local  governments 
have  also  invested  in  analytical  capability  (see 
app.  B).  Indeed,  one  of  the  largest  and  most  pro- 
ductive government  analysis  staffs  was  housed 
in  the  New  York  City  Health  Services  Adminis- 
tration. 

To  date,  Government  has  been  the  principal 
consumer  of  analysis.  Still,  most  observers  of 
CEA/CBA  consider  Government's  past  use 
quite  limited.  One  reviewer,  however,  has  con- 
cluded that  "there  is  evidence  to  suggest  that 


such  studies  have  played  an  important  role  in 
public  policy  determination"  (16).  According  to 
Dunlop,  two  examples  support  this  conclusion. 
First,  Enke's  mid-1960s  finding  that  the  benefits 
of  birth  control  exceeded  costs  by  a  factor  of  100 
contributed  to  Congress'  significant  expansion 
of  Agency  for  International  Development  funds 
to  assist  developing  countries  in  implementing 
birth  control  programs.  Second,  Dunlop  says, 
CBAs  of  water  fluoridation  have  "nearly  always 
preceded"  dental  health  program  development, 
with  the  studies  being  "widely  disseminated  to 
the  public"  prior  to  a  public  vote  on  fluorida- 
tion. 

Other  analysts  are  less  sanguine  about  the  ef- 
fects of  past  analyses  on  policy.  Jeffrey  Weiss, 
who  headed  the  Program  Analysis  staff  of  the 
New  York  City  Health  Services  Administration, 
has  concluded  that,  owing  to  political  and  budg- 
etary factors,  his  staff  analyses  had  little  impact 
on  broad  strategies.  Analyses  initiated  by  the 
staff  lacked  a  political  constituency  and  tended 
to  be  ignored.  Of  analyses  requested  by  city 
officials,  a  few  were  followed  by  policies  con- 
sistent with  their  findings,  but  these  tended  to 
support  policymakers'  predetermined  biases  on 
the  issues.  Only  when  administrators  had  not 
worked  through  problems  on  their  own,  and 
strong  political  constituencies  were  not  in- 
volved, could  analyses  affect  the  decisionmak- 
ing framework;  and  in  most  such  cases,  the  ef- 
fects were  "suboptimizing,"  for  example,  resolv- 
ing narrow  management  issues.  Weiss  has  cited 
a  couple  of  successes,  however — a  critical  anal- 
ysis of  the  neighborhood  family  care  program 
resulted  in  constriction  of  the  program;  and  a 
study  of  methadone  treatment  of  heroin  addicts 
created  a  strong  intellectual  rationale  for  drug 
detoxification  efforts  in  New  York  City  (355). 
Weiss  noted  that  the  city  administrator  might 
have  gone  ahead  with  the  latter  efforts  in  the  ab- 
sence of  the  analysis,  but  the  study  provided 
support  for  the  policy  action  (633). 
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Table  4.— Overview  of  Agency  Activities  in  Decisions  Concerning  Medical  Technology 


Decision  areas  and  activities 


Agency/mission 

R&D 

Evaluation 

Regulation 

Financing 

NIH 

Improve  human  health  by 
increasing  medical 
knowledge  and  encourag- 
ing the  development  of 
safe  and  effective  med- 
ical technologies 

Supports  and  conducts 
basic  and  applied 
research 

Establishes  standards  for 
research,  clinical  trials, 
and  human  experimen- 
tation 

Supports  and  conducts 
clinical  trials 

Sponsors  Consensus  De- 
velopment Conferences 
to  evaluate  medical 
technologies 

Evaluates  outcomes  of 
sponsored  research 

FDA 

Protect  the  American 
public  from  unsafe  and 
inefficacious  drugs  and 
medical  devices  and  un- 
safe foods  and  cos- 
metics 

- 

Establishes  regulatory  requ 
strations  of  the  safety  and 
medical  devices 

Evaluates  safety  and  efficac 
etc. 

rements  for  demon- 
efficacy  of  drugs  and 

y  data  from  clinical  trials, 

Regulates  market  ap- 
proval of  drugs  and 
medical  devices 

Regulates  drug  and  med- 
ical device  manufac- 
turing processes 

Monitors  the  safety  of 
marketed  drugs  and 
medical  devices 

NCHCT 

1  Inriprtakp  anH  Qiinnnrt 

ui  iuci  iar\c  01  iu  oup|JUi  i 

assessments  of  health 
care  technologies 

Lvaiuaico  oUbidi ,  e  1 1  h  l.  a  i  , 

legal,  medical,  scientif- 
ic, and  economic  as- 
pects of  new,  emerging, 
and  existing  medical 
technologies 
Compiles  and  dissemi- 
nates information  con- 
cerning the  safety  and 
efficacy  of  medical 
technologies 

MskpQ  rprnmmonfifl- 

l¥lfll\Cj  <  CLUI  Ul  1  ICI  lUu 

tions  concerning  re- 
imbursement issues 
submitted  by  HCFA 

NCHSR 

Support  health  services 
research  on  a  variety  of 
health  care  issues 

Supports  health  services  research,  primarily  on  new 
and  existing  technologies 

HCFA 

Administer  the  medicare/ 
medicaid  programs, 
PSRO  program,  and  sup- 
port research  and  sta- 
tistics efforts  for  these 
programs 

Supports  health  services  research  on  a  variety  of 
health  care  issues,  primarily  focused  on  areas  of  pro- 
grammatic responsibility 

Establishes  reimburse- 
ment criteria  for  new 
and  established  medical 
technologies 

Establishes  benefits 
packages  for  med- 
icare 

Makes  reimbursement 
decisions  concerning 
medical  technologies 

HSAsa 

Develop  and  implement 
local  health  plans  and 
monitor  the  dissemina- 
tion of  health  services 

Review  major  capital  ex- 
penditures for  cer- 
tificate of  need 

Review  use  of  Federal 
funds  for  certain  pro- 
grams 

PSROsb 

Assure  that  health  care 
services  paid  for  under 
certain  Federal  programs 
are  medically  necessary, 
meet  professionally  rec- 
ognized standards  of 
care,  and  are  provided  at 
the  most  economical  lev- 
el possible  consistent 
with  quality  care 
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quality  of  medical  care 
through  medical  care 
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ing medical  care 

aAt  the  national  level,  the  Health  Resources  Administration's  Bureau  of  Health  Planning  is  responsible  for  providing  the  community  or  regionally  based  HSAs  with 

technical  assistance  and  guidance  regarding  the  planning  and  delivery  of  health  care  services. 
DThe  Health  Standards  and  Quality  Bureau  of  HCFA,  DHHS,  is  the  Federal  level  organization  that  provides  general  guidance  and  technical  assistance  to  the  locally 

controlled  and  operated  PSROs. 


SOURCE:  Office  of  Technology  Assessment. 
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Figure  2.— Department  of  Health  and  Human  Services— Organizational  Components  Involved 
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5. 

Reimbursement 


INTRODUCTION  AND  BACKGROUND 


Medicare  and  Blue  Cross/Blue  Shield,  the 
largest  government  and  private  health  insurance 
programs,  respectively,  pay  for  a  substantial 
share  of  the  Nation's  total  health  care  outlay.  In 
1978,  for  example,  medicare  financed  24  percent 
of  all  hospital  care  and  16  percent  of  all  physi- 
cians' services.  In  fact,  its  payments  represented 
more  than  15  percent  of  all  personal  medical  ex- 
penditures in  the  United  States.  In  the  same 
year,  Blue  Cross/Blue  Shield  paid  for  nearly 
11.5  percent  of  all  personal  health  care  expendi- 
tures (84).  Through  financing,  both  programs 
can  affect  the  rates  at  which  new  technologies 
are  developed,  diffused,  and  utilized,  and  at 
which  inefficacious,  outmoded,  and  unsafe  serv- 
ices are  phased  out. 

This  chapter  focuses  on  the  medicare  program 
and  its  reimbursement  coverage  process.  Also 
included  is  a  discussion  of  Blue  Cross  and  Blue 
Shield  and  its  reimbursment  coverage  process. 
The  latter  discussion  is  included  because  the  na- 
tional Blue  Cross  and  Blue  Shield  Associations 


and  the  affiliated  plans  (the  actual  administra- 
tive units,  who  are  relatively  autonomous  and 
who  receive  guidance  but  not  mandatory  proce- 
dures from  the  national  associations)  represent 
the  largest  nongovernmental  third-party  payer 
in  the  Nation  and  because  their  processes  of  de- 
ciding on  coverage  of  medical  technologies  for 
reimbursement  are  similar  to  medicare's. 

Medicare  is  a  nationwide,  federally  adminis- 
tered health  insurance  program  authorized  in 
1965.  It  provides  benefits  for  people  over  age 
65,  for  certain  individuals  eligible  for  disability 
payments,  and  for  certain  individuals  who  need 
kidney  transplantation  or  dialysis.  The  medic- 
aid program  is  a  Federal  program  that  is  admin- 
istered individually  by  each  participating  State 
government.  Each  State  can  use  its  own  proce- 
dures for  coverage  decisions.  Although  medic- 
aid is  not  covered  in  this  chapter,  in  concept, 
many  of  the  arguments  presented  would  apply 
equally  well  to  that  program. 


THE  MEDICARE  COVERAGE  PROCESS 


Section  1962  of  the  Social  Security  Act  man- 
dates that  medicare  shall  pay  only  for  medical 
services  that  are  "reasonable  and  necessary"  for 
diagnosis,  treatment,  or  improved  functioning.1 
By  granting  Government  officials  the  authority 
to  determine  which  new  and  existing  services2 
are  eligible  for  reimbursement,  this  section  of 
the  law  involves  medicare  in  technology  deci- 
sions. Medicare  has  refrained  from  establishing 
a  definitive  interpretation  of  the  "reasonable 

'Sec.  1862. (a)  Notwithstanding  any  other  provisions  of  this  title, 
no  payment  may  be  made  under  part  A  or  part  B  for  any  expenses 
incurred  for  items  or  services  (1 )  which  are  not  reasonable  and  nec- 
essary for  the  diagnosis  or  treatment  of  illness  or  injury  or  to  im- 
prove the  functioning  of  a  malformed  body  member. 

2The  Social  Security  Act  also  specifically  excludes  certain  serv- 
ices from  payment  eligibility. 


and  necessary"  language  of  the  Social  Security 
Act  and  relied  on  a  loosely  structured  and  de- 
centralized mechanism  to  provide  coverage 
guidelines. 

The  medicare  program  is  administered  in  two 
parts:  part  A,  financed  by  payroll  taxes,  pro- 
vides reimbursement  for  services  in  hospitals, 
extended  care  facilities,  and  other  institutions; 
and  part  B,  a  voluntary  supplementary  pro- 
gram, pays  for  physicians'  services.  Under  the 
present  coverage  mechanism,  the  contractors 
who  administer  part  A,  fiscal  intermediaries, 
and  part  B,  carriers,  of  medicare  bear  the  ini- 
tial   responsibility   for    identifying  coverage 
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issues  and  determining  reimbursement  policy.3 
Through  the  advice  provided  by  their  medical 
advisors,  contractors  make  coverage  determina- 
tions about  the  majority  of  new  services  that 
they  identify  (98). 

When  the  contractors  feel  unable  to  make  a 
judgment  on  a  particular  coverage  issue,  they 
submit  the  issue  to  one  of  10  medicare  regional 
offices.  As  a  rule,  referrals  are  made  by  carriers, 
who  process  physicians'  bills,  rather  than  by 
fiscal  intermediaries,  who  process  hospital  bills. 
This  is  because  under  the  cost  reimbursement 
system,  hospital  bills  generally  are  not  exam- 
ined for  questions  about  the  use  of  a  particular 
technology.  In  fact,  most  hospital  billing  forms 
provide  little  specific  information  about  the 
various  elements  of  service  (398,574).  Although 
intermediaries,  as  well  as  hospitals,  physicians, 
and  the  manufacturers  of  drugs  and  devices, 
may  occasionally  raise  a  coverage  issue,  carriers 
usually  perform  this  function. 

Coverage  decisions  by  both  contractors  (239) 
and  regional  offices  (398)  appear  to  be  based 
primarily  on  two  related  criteria:  1)  the  tech- 
nology's stage  of  development,  and  2)  its  gener- 
al acceptance.  If  a  new  technology  is  perceived 
to  have  moved  beyond  experimental  status 
toward  full  clinical  application  and  to  be  ac- 
cepted by  the  local  medical  community,  then  it 
is  deemed  "reasonable  and  necessary."  These 
criteria,  however,  lack  precise  standards,  and 
the  contractors'  procedures  for  considering 
them  tend  to  be  informal  and  highly  variable. 
Thus,  for  example,  a  medical  advisor  may  base 
a  decision  on  immediate  personal  knowledge  of 
a  technology's  stage  of  development  and  accept- 
ance or  may  survey  the  literature  and  seek  out 
the  opinions  of  other  medical  consultants,  local 
specialty  society  representatives,  advocates  of 
the  procedure,  and  the  advisors  of  other  area  in- 
surance programs. 

In  addition,  both  contractors  and  medicare 
regional  offices  appear  to  show  considerable 
variation  in  the  priority  they  accord  to  coverage 
questions  and  their  approach  to  handling  them. 
Some  regional  offices  will  attempt  to  resolve 


'Medicare  contractors  for  part  A  deal  with  hospital  services  and 
contractors  for  part  B  deal  with  physicians'  services. 


many  of  the  issues  referred  to  them  by  seeking 
out  the  opinions  of  local  contractors;  others  will 
tend  to  transmit  such  issues  directly  to  the 
medicare  central  office  (398,291).  Similarly, 
within  a  region,  some  carriers  will  display  far 
more  initiative  than  others  in  identifying  a 
coverage  issue,  pursuing  information  about  it, 
and  making  a  decision  (398).  As  a  result,  the 
specific  package  of  benefits  for  which  medicare 
will  provide  reimbursement  varies  somewhat 
across  the  country,  and  there  is  no  national 
standard  for  covered  services. 

When  the  contractor  and  medicare  regional 
office  are  unable  to  resolve  an  issue,  it  is  re- 
ferred to  the  Health  Care  Financing  Adminis- 
tration (HCFA).4  This  agency,  in  turn,  may  re- 
quest a  coverage  recommendation5  from  the 
Public  Health  Service  (PHS).6  Historically,  PHS 
generated  its  responses  through  an  informal  and 
loosely  structured  procedure.  Typically,  one, 
two,  or  three  professional  PHS  staff  members 
assigned  to  the  task  researched  a  coverage  ques- 
tion by  attempting  to  survey  the  relevant  medi- 
cal literature  and  consult  with  appropriate  ex- 
perts. This  process  has  been  inherently  unsys- 
tematic, because  the  small  PHS  staff  has  lacked 
the  benefit  of  established  channels  of  communi- 
cations to  medical  specialty  groups  and  to  other 
PHS  agencies  such  as  the  National  Institutes  of 
Health  (NIH)  and  the  Food  and  Drug  Adminis- 
tration (FDA)  (98). 

Within  this  ad  hoc  framework,  PHS  has  tradi- 
tionally applied  four  criteria  to  coverage  recom- 

4Reimbursement  coverage  questions  are  referred  to  the  Health 
Standards  and  Quality  Bureau  (HSQB)  in  HCFA.  Prior  to  the  crea- 
tion of  HCFA,  they  were  referred  to  medicare's  Division  of  Pro- 
vider and  Reimbursement  Policy. 

5ln  1977,  the  Administrator  of  HCFA  and  the  Assistant  Secre- 
tary of  Health  entered  into  an  agreement  which  formalized  the 
Public  Health  Service's  role  in  providing  coverage  recommenda- 
tions. Since  the  late  1960  s,  the  medicare  program's  lack  of  medical 
advisors  has  resulted  in  an  almost  automatic  referral  of  coverage 
questions  to  PHS,  although  occasionally  HCFA  has  tried  to  group 
related  issues  together  or  resolve  a  matter  which  seems  to  warrant 
little  attention  because  it  has  either  been  previously  raised  and 
answered  by  PHS  or  it  concerns  an  apparently  worthless  service 
(e.g.,  colonic  irrigation)  or  a  well-established  and  accepted  one. 

The  current  PHS  unit  providing  coverage  recommendations  is 
the  National  Center  for  Health  Care  Technology  (NCHCT).  The 
Center  was  created  by  Public  Law  95-623  in  November  1978.  Be- 
fore the  formation  of  NCHCT,  the  Office  of  Health  Practice  As- 
sessment bore  this  responsibility.  Previously,  it  was  the  function  of 
the  Bureau  of  Quality  Assurance  in  the  Health  Services  Adminis- 
tration. 
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mendations:  1)  safety,  2)  efficacy,  3)  stage  of 
development,  and  4)  acceptance  by  the  medical 
community.  Although,  these  criteria  have  re- 
cently been  made  more  explicit7  through  a  PHS 
staff  paper  on  coverage  policy,8  they  still  lack 
precise  definition. 

PHS  recommendations  generally  have  not  at- 
tempted to  recommend  specific  indications  for 
use  of  technologies.  The  PHS  staff  have  main- 
tained that  coverage  policies  should  be  ex- 
pressed in  broad  terms  and  questions  of  appro- 
priate usage  for  a  set  of  specific  patient  indica- 
tions should  be  addressed  by  the  practicing 
medical  community  and  the  Professional  Stand- 
ards Review  Organizations.  Thus,  a  PHS  de- 
scription of  its  coverage  policy  notes  (98): 

The  coverage  system  cannot  attempt  to  be  an 
encyclopedic  listing  of  which  interventions  to 
use  in  which  circumstances  nor  serve  as  a  kind  of 
substitute  for  medical  education  and  clinical 
training.  Rather,  the  coverage  system  uses  broad 
strokes  to  sketch  the  boundaries  of  accepted 
good  medical  practice,  and  leaves  the  fine-tun- 
ing of  the  system  to  the  "back-end"  mechanism, 
the  Professional  Standards  Review  Organiza- 
tions (PSROs). 

Recently,  however,  HCFA  has  shown  sOme  re- 
luctance in  accepting  PHS  recommendations 
and  sought  to  issue  guidelines  which  cite  indica- 
tions for  use  more  often. 

After  the  PHS  staff  develop  a  coverage  rec- 
ommendation, they  send  a  memorandum  to 
HCFA's  Health  Standards  and  Quality  Bureau 
(HSQB).  This  Bureau  has  usually  accepted  the 

'Historically,  these  criteria  have  been  developed  and  utilized  im- 
plicitly (33). 

"A  PHS  staff  paper  provides  the  elaboration  of  these  terms  (98): 

1.  Safety:  What  is  the  likelihood  and  severity  of  risk  associated 
with  the  technology? 

2.  Efficacy:  What  is  the  likelihood  that  the  technology  will  pro- 
duce the  desired  health  benefit?  No  distinction  is  drawn  here 
between  efficacy  and  effectiveness,  although  in  other  con- 
texts, "efficacy"  is  used  to  mean  the  probability  that  a 
modality  will  produce  a  desired  outcome  under  ideal  circum- 
stances, while  "effectiveness"  is  used  to  mean  that  the  desired 
outcome  will  result  under  average  clinical  conditions.  The 
term  "efficacy"  as  used  here  is  intended  to  convey  a  hybrid  of 
both  meanings,  since  the  data  base  for  coverage  decisions  is 
sometimes  derived  from  laboratory  or  clinical  trial  environ- 
ments and  sometimes  from  routine  practice  environments. 

3.  Stage  of  development:  Has  the  technology  moved  beyond 
the  experimental  stage  to  full  clinical  application? 

4.  Acceptance  by  the  medical  community. 


policy  suggested  by  PHS  and  has  issued  appro- 
priate guidelines  to  medicare  regional  offices 
and  contractors  (282).  Because  of  the  large 
volume  of  services  involved,  policy  implemen- 
tation depends  mainly  on  the  good  faith  of  pro- 
viders and  on  the  threat  of  potential  audits. 

Cost  and  Efficacy 

Although  the  cost  of  medical  technology  has 
focused  attention  on  reimbursement,  cost  infor- 
mation has  not  been  explicitly  or  directly  con- 
sidered in  medicare  coverage  determinations.  In 
a  few  cases,  where  PHS  has  concluded  that  a 
new  technology  is  a  modification  of  an  existing 
intervention  and  offers  little  additional  benefit, 
it  has  favored  coverage  but  recommended  pay- 
ment at  the  same  level  as  that  for  the  established 
service.  This  type  of  judgment  represents  a 
weighing  of  marginal  cost  and  marginal  value, 
so  it  might  be  said  that  PHS  has  conducted  im- 
plicit cost-effectiveness  analyses  (CEAs).  The 
practice  of  paying  usual  and  customary  fees, 
however,  does  not  easily  accommodate  such 
analyses. 

Unlike  cost,  efficacy  is  one  of  the  four  criteria 
PHS  has  traditionally  applied  to  coverage  ques- 
tions. Several  recent  PHS  procedural  changes 
have  promoted  more  systematic  examination  of 
efficacy  data:  1)  the  development  of  communi- 
cation ties  with  NIH,  FDA,  and  medical  special- 
ty societies;  2)  the  formal  linking  of  FDA  drug 
and  medical  device  policies  to  PHS  coverage  de- 
terminations (98);  and  3)  the  setting  of  a  prece- 
dent in  the  computed  tomography  (CT)  body 
scan  decision  to  restrict  coverage  to  uses  that  are 
supported  by  current  evidence  of  efficacy  (282), 
although  the  list  of  indications  for  those  scans 
appears  to  include  nearly  every  possible  use 
(389). 

The  extent  to  which  these  procedural  changes 
will  ensure  a  greater  emphasis  on  efficacy  re- 
mains uncertain.  Despite  the  CT  body  scan  rec- 
ommendation, PHS  has  generally  not  attempted 
to  specify  indications  for  use,  contending  it 
would  do  so  only  in  the  consideration  of  drugs 
or  in  an  exceptional  case  involving  a  high-risk 
or  a  high-cost  technology.  Although  it  is  com- 
mitted to  reflecting  FDA's  safety  and  efficacy 
rulings  by  following  FDA's  decisions  on  ap- 
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proved  and  contraindicated  drug  use,  PHS  exer- 
cises discretion  in  considering  uses  for  indica- 
tions on  which  FDA  has  not  ruled.  Further,  drug 
coverage  questions  are  not  usually  raised,  be- 
cause medicare  only  pays  for  the  use  of  drugs  in 
hospitals,  and  it  is  difficult  for  fiscal  intermedi- 
aries to  identify  individual  hospital  service  com- 
ponents.9 Questions  about  medical  equipment 
are  far  more  typical,  but  the  impact  of  relating 
coverage  recommendations  to  FDA  determina- 
tions is  unclear,  because  the  Medical  Devices 
Amendments  of  1976  are  still  in  the  process  of 
being  implemented  (see  ch.  8).  Perhaps  more 
importantly,  adequate  evidence  of  the  efficacy 
and  safety  of  devices  and  procedures  is  often  not 
available  (465),  and  in  the  absence  of  such  evi- 
dence, technologies  judged  to  be  nonexperimen- 
tal  and  accepted  usually  have  received  a  positive 
coverage  recommendation  from  PHS. 

Application  of  efficacy  criteria  also  is  limited 
by  the  selected  number  of  coverage  questions 
that  reach  the  PHS  agenda.10  The  current  sys- 
tem, in  which  issues  may  be  identified  by  fiscal 
intermediaries  when  they  process  hospitals'  bills 
and  by  carriers  when  they  process  physicians' 
bills,  is  basically  reactive.  Identification  of  new 
services  can  be  difficult,  however,  because  in- 
formation may  be  lost  or  obscured  not  only  by 
classification  into  service  code  categories,  but 
also  by  translation  from  provider  to  intermedi- 
ary codes,  although  the  tendency  to  attach  a 
higher  cost  to  a  new  technology  serves  as  a  ma- 
jor screening  aid  (399,569).  Helen  Smits,  the  Di- 
rector of  HCFA's  HSQB  has  observed  (575): 

A  consistent  method  of  coding  diagnoses  and 
procedures,  consistently  applied,  is  essential  to 
any  real  "capture"  of  new  technologies  by  reim- 
bursement .  .  .  Even  when  a  single  procedural 
code  has  been  agreed  upon  and  put  into  use, 
problems  on  variable  coding  are  likely  to  persist 
and  to  make  accurate  application  of  coverage 
decisions  very  difficult. 

Identification  of  a  new  hospital  service  is  par- 
ticularly problematic.  This  is  because,  as  noted 

'It  is  also  extremely  difficult  to  monitor  drug  use  for  approved 
indications. 

10While  judgments  made  by  contractors  and  the  regional  office 
about  acceptance  and  stage  of  development  may  be  influenced  by 
efficacy  data,  the  focus  appears  to  be  more  directly  on  stage  of  de- 
velopment and  acceptance. 


above,  fiscal  intermediaries  generally  do  not  ex- 
amine hospital  bills  for  coverage  questions  and 
the  majority  of  billing  forms11  provide  little  in- 
formation about  the  use  of  particular  technol- 
ogy. Certificate-of-need  requirements,  end  of 
the  year  or  prospective  reimbursement  negotia- 
tion, and  informal  discussions  between  inter- 
mediaries and  the  hospitals  in  their  area  appear 
to  alleviate  this  problem  only  partially. 

Another  way  that  a  coverage  issue  may  be 
raised  is  through  the  interaction  of  intermedi- 
aries and  hospital  administrators  or  staff  when 
the  hospital  is  planning  to  acquire  or  to  offer  a 
new  service.  Identification  of  large-scale,  dis- 
crete, and  potentially  expensive  technologies 
may  be  accomplished  more  easily  through  this 
type  of  interaction  than  it  is  through  billings. 
Neither  the  extent  nor  the  potential  usefulness  of 
such  an  identification  method,  however,  was 
analyzed  by  OTA. 

Even  when  a  new  service  is  identified,  the 
coverage  decision  often,  perhaps  usually,  will 
be  made  by  the  contractor  or  intermediary  and 
will  not  be  brought  to  the  attention  of  HCFA  or 
PHS  unless  a  negative  determination  is  legally 
challenged  by  a  physician,  hospital,  or  patient. 
For  example,  PHS  has  never  been  asked  to  make 
a  judgment  about  the  coverage  of  coronary  ar- 
tery bypass  grafting  (470),  an  expensive  and 
widely  performed  procedure.  In  addition,  the 
PHS  coverage  agenda  is  severely  circumscribed 
by  an  almost  exclusive  emphasis  on  new  tech- 
nology. The  one  major  exception  to  this  pattern 
occurred  in  1977,  when  PHS  issued  recommen- 
dations about  28  established  procedures  after 
Blue  Shield  had  concluded  they  were  outmoded 
or  ineffective  and  should  be  excluded  from  rou- 
tine reimbursement  payments  (431,432,433). 

Currently,  however,  both  HCFA  and  PHS  are 
studying  a  number  of  possible  changes  in  the 
medicare  coverage  process  that  may  increase  the 


"Because  of  the  diversity  of  elements  involved  in  surgical  proce- 
dures, identification  of  a  "new"  surgical  procedure  may  be  particu- 
larly difficult.  For  example,  David  Eddy  notes,  "In  surgery,  it  can 
be  very  difficult  to  identify  when  a  procedure  is  "new"  or  suffi- 
ciently different  from  other  procedures  to  require  a  new  evalua- 
tion .  .  .  frequently  a  procedure  can  be  described  only  as  a  set  of 
maneuvers,  and  these  maneuvers  can  change  in  subtle  but  impor- 
tant ways"  (167). 
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weight  given  to  efficacy  data,  as  well  as  formal- 
ly introduce  cost  criteria.  HCFA  is  examining 
several  possible  actions:  utilizing  cost  as  a 
coverage  criterion,12  implementing  regulations 
that  would  formally  define  the  "reasonable  and 
necessary"  language  of  the  Social  Security 
Act,13  establishing  a  uniform  service  code,14  and 
issuing  more  guidelines  which  relate  coverage  to 
appropriate  indications  for  use.15  At  the  same 
time,  the  new  PHS  coverage-recommending 
unit,  the  National  Center  for  Health  Care  Tech- 
nology (NCHCT)  is  considering  utilizing  three 
additional  coverage  criteria:  conformity  to 
health  planning  guidelines,  relative  efficacy, 
and  cost  effectiveness.  Reservations  have  been 
raised  about  the  introduction  of  the  latter  two 
criteria,  however,  because  of  methodological 
difficulties  in  measurement.  As  a  result,  the  PHS 
coverage  staff  are  currently  awaiting  the  results 
of  NCHCT's  experience  with  applying  CEA  in 
its  comprehensive  assessment  activities.  Ac- 
cording to  a  PHS  staff  paper  (98): 

The  chief  difficulty  is  how  to  measure  relative 
efficacy  and  cost-effectiveness  in  an  operational 
way.  If  one  is  comparing  the  relative  efficacy  of 
two  modalities,  how  much  more  efficacious 
must  the  more  expensive  or  more  risky  technol- 
ogy be  in  order  to  be  "worth"  the  added  cost  or 
risk?  The  techniques  for  comparing  relative  ef- 
ficacy and  determining  cost-effectiveness  are  less 
reliable  than  the  methods  for  assessing  safety 
and  efficacy,  and  our  experience  in  using  the 
outputs  of  relative  efficacy  and  cost-effective- 
ness studies  as  a  basis  for  policy  decisions  is  very 
weak.  It  would  appear  that  some  additional  de- 
velopmental work  is  needed  before  relative  ef- 
ficacy and  cost-effectiveness  can  be  applied 
routinely  as  criteria  for  coverage  recommenda- 
tions. One  way  to  demonstrate  and  test  the  ap- 
plication of  these  two  criteria  would  be  to  use 
them  in  the  course  of  the  large-scale  evaluations 


12The  Office  of  the  General  Counsel  has  been  asked  to  determine 
whether  the  "reasonable  and  necessary"  language  of  sec.  1862  and 
the  reference  to  "reasonable  cost"  in  other  parts  of  the  legislation 
permit  introduction  of  cost  criteria  into  coverage  decisions  (292). 

13HCFA's  Division  of  Provider  and  Reimbursement  Policy  has 
drawn  up  a  set  of  draft  regulations  defining  "reasonable  and  nec- 
essary," and  the  Office  of  Policy  Planning  and  Research  also  is  in 
the  process  of  performing  the  same  task. 

14A  uniform  code  also  may  be  used  to  produce  a  list  of  services 
that  are  covered  (399). 

15In  recent  months,  HCFA  has  shown  greater  interest  in  issuing 
guidelines  that  cite  indications  for  use. 


to  which  NCHCT  will  subject  the  high  priority 
technologies.  If  the  "bugs"  appear  to  have  been 
worked  out  in  the  mega-assessments,  considera- 
tion would  then  be  given  to  applying  relative  ef- 
ficacy and  cost-effectiveness  more  routinely  in 
coverage  decisions. 

Coverage  Reevaluation 

Cost,  safety,  efficacy,  and  legal  concerns 
have  all  contributed  to  the  current  reevaluation 
of  medicare  coverage  decisions.  Most  impor- 
tant, perhaps,  the  rising  cost  of  medical  care  has 
confronted  policymakers  with  the  need  to  con- 
tain health  expenditures  and  rationally  allocate 
resources.  Increases  in  health  care  expenditures 
associated  with  both  the  enactment  of  medicare 
and  the  rapid  diffusion  and  use  of  technology 
have  generated  additional  interest  in  creating 
cost  control  mechanisms. 

Although  the  overall  impact  of  technological 
innovation  on  health  care  spending  is  unclear 
(17),  it  is  apparent  that  economic  incentives 
strongly  favor  the  spread  of  technology  (196, 
528).  Because  medicare  reimburses  retrospec- 
tively, it  provides  an  open-ended  commitment 
to  pay  for  covered  services.  Under  medicare  and 
some  other  third-party  insurance,  hospitals  (the 
most  expensive  element  of  the  health  care  sys- 
tem) are  reimbursed  on  the  basis  of  costs;  physi- 
cians are  reimbursed  on  the  basis  of  charges; 
and  patients  are  partly  insulated  from  immedi- 
ate actual  costs.  As  a  result,  services  may  be  uti- 
lized even  when  patient  outcome  benefits  are 
marginal  or  uncertain  (468). 

The  tendency  toward  utilization  of  services  is 
also  encouraged  by  other  factors:  competition 
among  hospitals  to  achieve  quality  and  prestige 
and  attract  patients  and  physicians;  public  de- 
mand for  sophisticated  technology;  practition- 
ers' desire  to  do  the  most  possible  for  their  pa- 
tients and  to  achieve  a  high  degree  of  certainty 
in  their  judgments  so  as  to  avoid  malpractice 
suits;  physician  specialization;  and  the  stress  on 
ancillary  services  (470,546). 

Spiraling  medical  expenditures  and  their  asso- 
ciation with  a  reimbursement  system  that  pro- 
motes technology  development  and  use  has  en- 
couraged Government  officials  to  reevaluate 
medicare's  traditional  social  insurance  orienta- 
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Hon.  At  the  time  of  the  enactment  of  the  legisla- 
tion in  1965, 16  and  in  the  years  since,  there  has 
been  an  emphasis  on  accommodation  with  the 
existing  health  care  system  (191). 17  This  theme  is 
reflected  in  the  nature  of  the  present  coverage 
mechanism:  the  heavy  reliance  on  fiscal  inter- 
mediaries and  carriers,  the  failure  to  formally 
define  the  "reasonable  and  necessary"  language 
of  section  1862,  and  the  tendency  for  PHS  to 
react  favorably  to  the  majority  of  coverage 
questions  it  has  addressed.  But  as  cost  contain- 
ment becomes  an  increasingly  important  objec- 


16The  Social  Security  Act  states  in  sec.  1801: 

Nothing  in  this  title  shall  be  construed  to  authorize  any  federal  of- 
ficer or  employee  to  exercise  any  supervision  or  control  over  the  prac- 
tice of  medicine  or  the  manner  in  which  medical  services  are  pro- 
vided .  .  . 

For  a  political  history  of  medicare,  see  R.  Harris,  A  Sacred  Trust 
(278);  T.  H.  Marmor  and  J.  Marmor,  The  Politics  of  Medicine 
(381),  and  O.  W.  Anderson,  The  Uneasy  Equilibrium:  Private  and 
Public  Financing  of  Health  Services  in  the  United  States,  "Uneasy,  " 
1875-1965  (25). 

l7Of  course,  the  medicare  legislation  encouraged  cost  contain- 
ment through  physician  self-regulation  by  originally  establishing 
hospital  utilization  review  committees  in  1965  and  later  creating 
PSROsin  1972. 


tive,18  interest  in  medicare's  becoming  a  more 
selective  purchaser  of  health  services  is  rising 
(34).  Within  this  changing  perspective,  cost-ef- 
fectiveness information  would  seem  to  possess 
substantial  appeal. 

Reexamination  of  the  medicare  coverage  sys- 
tem also  stems  from  a  growing  awareness  that 
technological  innovation  and  health  status  do 
not  always  seem  to  be  directly  related  and  that 
the  safety  and  efficacy  of  many  technologies 
have  not  been  adequately  evaluated  (238). 
There  is  interest  in  protecting  patients  from 
risky,  ineffective,  or  unproven  services.  The 
result  has  been  that  coverage  decisions  are 
becoming  more  closely  tied  to  available  safety 
and  efficacy  information. 

In  addition,  legal  requirements  have  spurred 
PHS  and  HCFA  review  of  the  current  coverage 
process.  As  the  likelihood  of  more  coverage  de- 
nials increases,  there  is  a  greater  need  to  estab- 
lish a  firmer  legal  basis  for  decisions  (291,574). 

laFeder  suggests  that  reorganization  was  necessary  for  a  shift  in 
focus  from  social  insurance  to  health  policy  (191).  Several  HCFA 
officials  who  were  interviewed  possessed  a  similar  perspective. 


THE  BLUE  CROSS/BLUE  SHIELD  COVERAGE  PROCESS 


Cost  considerations,  along  with  safety  and  ef- 
ficacy and  legal  concerns,  in  addition  to  leading 
medicare  to  reevaluate  its  coverage  policies, 
have  led  nongovernmental  insurers  to  reevalu- 
ate their  coverage  policies.  Traditionally,  these 
insurers'  interest  in  escalating  health  care  costs 
has  been  moderated  by  their  ability  to  respond 
by  raising  insurance  premiums.  But  as  the  pur- 
chasers of  health  insurance — management,  la- 
bor unions,  and  individual  subscribers — have 
become  more  resistant  to  higher  premiums 
(470),  third-party  payers  have  been  increasingly 
confronted  with  the  need  to  contain  health  care 
costs  and  become  more  selective  purchasers  of 
health  services. 

Blue  Cross  and  Blue  Shield,  the  largest  non- 
Government  insurers,  with  over  40  percent  of 
the  Nation's  subscribers,  have  called  for  cost 
containment  initiatives  since  the  early  1970's. 
Over  the  years,  they  have  pursued  a  wide  range 
of  policies,  including  utilization  review  of  length 


of  stay  and  level  of  care,  health  planning,  bene- 
fit package  designs  that  emphasize  low-cost  op- 
tions (such  as  outpatient  surgery),  alternative 
delivery  systems,  consumer  education,  and  cov- 
erage reevaluation  (415).  Like  medicare,  how- 
ever, Blue  Cross  and  Blue  Shield  traditionally 
have  refrained  from  issuing  many  negative  cov- 
erage guidelines.  In  part,  this  probably  arises 
from  a  historical  social  insurance  orientation,  as 
well  as  Blue  Cross  and  Blue  Shield's  desire  to:  1) 
remain  competitive  with  commercial  insurers, 
2)  fulfill  the  expectations  of  beneficiaries,  hospi- 
tals, and  physicians  that  services  will  be  covered 
by  insurance  policies,  and  3)  forestall  legal  chal- 
lenges that  might  result  from  denial  of  payment. 

Blue  Cross  and  Blue  Shield's  treatment  of  cov- 
erage issues  parallels  medicare's  in  many  re- 
spects. Like  medicare,  Blue  Cross  and  Blue 
Shield  make  the  majority  of  coverage  decisions 
through  a  decentralized  and  loosely  structured 
process  that  places  key  emphasis  on  two  cover- 
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age  criteria:  stage  of  development  and  accept- 
ance by  the  medical  community.  Blue  Cross  and 
Blue  Shield  also  lack  a  uniform  national  benefit 
package;  interpretations  of  what  qualifies  as  a 
covered  service  vary  somewhat  throughout  the 
country.19 

Most  individual  Blue  Cross  and  Blue  Shield 
plan  contracts  exclude  "care  which  is  not  a  part 
of  generally  accepted  medical  practice"  and  "un- 
necessary or  inappropriate  care,"  although  the 
specific  wording  employed  in  different  contracts 
varies  (255).  These  criteria  lack  precise  defi- 
nition, however,  so  the  medical  advisors  and 
their  staffs  possess  considerable  discretion  in  ap- 
plying them,  although  coverage  questions  that 
are  viewed  as  particularly  important  may  be 
brought  to  the  attention  of  a  plan's  board  of  di- 
rectors. Medical  advisors  may  make  judgments 
based  on  immediate  personal  knowledge  or  may 
survey  the  literature  and  consult  with  advocates 
of  the  procedure,  local  specialty  societies,  the 
county  medical  society,  and  other  area  insurers. 
If  the  plan  functions  as  an  intermediary  for 
medicare  or  an  agent  for  medicaid  or  the  armed 
services  CHAMPUS  program,  the  medical  advi- 
sor will  generally  review  the  coverage  determi- 
nations of  these  programs.  Similarly,  the  medi- 
cal advisor  will  usually  study  the  coverage  rec- 
ommendations of  the  national  Blue  Cross  and 
Blue  Shield  Associations  and  other  member 
plans,  although  he  or  she  may  or  may  not  fol- 
low any  previous  rulings. 

The  national  Blue  Cross  and  Blue  Shield  As- 
sociations are  a  federation  of  137  State  and  local 
nonprofit  Blue  Cross  and  Blue  Shield  affiliates. 
Power  to  make  policy  ultimately  lies  with  the 
individual  plans,  but  the  national  organization 
exercises  considerable  leverage  through  the 
prominence  of  its  leadership  and  the  ability  to 
serve  as  a  public  spokesman  for  member  plans 
(141,255,415).  The  national  associations'  cover- 
age recommendations  are  made  by  the  Medical 
Necessity  Program  and  the  Medical  Advisory 
Committee. 

The  Medical  Necessity  Program,  developed  in 
1977  in  conjunction  with  the  American  College 


"While  benefit  package  coverage  variation  allows  responsive- 
ness to  local  patterns  of  practice,  it  also  raises  the  issue  of  equity. 


of  Physicians,  the  American  College  of  Sur- 
geons, and  the  American  College  of  Radiology, 
was  designed  to  curtail  reimbursement  for  out- 
moded, duplicative,  or  unproven  procedures. 
According  to  a  1977  Blue  Shield  bulletin  (433): 

The  Program  is  an  effort  to  align  the  objec- 
tives of  several  national  professional  medical  so- 
cieties and  of  Blue  Shield  in  improving  the  quali- 
ty of  care  while  simultaneously  reducing  the 
costs  of  procedures  that  are,  in  many  instances, 
outmoded,  redundant  in  combination  with 
others,  unlikely  to  yield  additional  information 
through  repetition,  or  of  unproven  value. 

Since  the  inception  of  this  program,  the  Blue 
Cross  and  Blue  Shield  Associations  have  recom- 
mended discontinuation  of  routine  payment  for 
68  surgical  and  diagnostic  procedures  and  hos- 
pital admission  battery  tests  (58,59,140,431, 
432,433).  While  physicians  may  still  order  these 
procedures,  they  must  provide  written  justi- 
fication for  their  use  in  order  to  be  reimbursed 
by  plans  that  adopt  the  medical  necessity  pro- 
posals. Most  plans  appear  to  be  accepting  these 
recommendations,  and  it  has  been  estimated 
that  full  implementation  may  result  in  annual 
savings  of  as  much  as  $200  million  from  limiting 
routine  hospital  admissions  tests  and  as  much  as 
$100  million  from  curtailing  the  use  of  68  surgi- 
cal and  diagnostic  procedures.20 

Moreover,  the  national  Blue  Cross  and  Blue 
Shield  Associations  make  over  100  coverage 
recommendations  a  year  through  the  delibera- 
tions of  their  Medical  Advisory  Committee. 
With  the  advice  of  medical  specialty  groups,  the 
Medical  Advisory  Committee  issues  suggested 
coverage  guidelines.  Usually  these  recommen- 
dations concern  new  technology  and  are  ex- 
pressed in  broad  terms,  but  sometimes  they 
specify  indications  for  use.  In  addition,  the  com- 
mittee serves  as  a  clearinghouse  for  information 
about  the  coverage  policies  of  each  of  the  Blue 
Cross  and  Blue  Shield  plans. 

The  national  Blue  Cross  and  Blue  Shield  As- 
sociations consider  only  a  limited  number  of 
coverage  questions.  Medical  specialty  group 

"In  some  areas  where  Blue  Cross  and  Blue  Shield  plans  are  in- 
corporated together,  they  share  the  same  medical  advisory  staff;  in 
other  areas,  they  operate  as  separate  organizations  with  separate 
staffs. 
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representatives  suggest  which  established  proce- 
dures should  be  examined  by  the  Medical  Neces- 
sity Program,  and  the  Medical  Advisory  Com- 
mittee's agenda  is  largely  determined  by  ques- 
tions raised  by  plans  when  they  seek  the  nation- 
al associations  advice. 

Cost  and  Efficacy 

Blue  Cross  and  Blue  Shield,  like  medicare, 
tend  not  to  consider  cost  information  directly  in 
coverage  deliberations.  In  a  few  cases,  where  a 
new  procedure  has  been  identified  as  a  modifi- 
cation of  an  existing  service  which  offers  little 
additional  benefit,  however,  payment  for  the 
new  procedure  has  been  fixed  at  the  same  level 
as  that  for  the  established  technology.  In  addi- 
tion, cost  has  traditionally  been  one  of  the  fac- 
tors examined  in  benefit  package  design.  Re- 
cently, CEA  was  used  in  the  development  of  a 
screening  program  when  the  National  Cancer 
Institute  contracted  with  the  Blue  Cross  and 
Blue  Shield  Associations  in  1976  to  develop  a 
model  prepaid  health  service  benefit  package  for 
cancer  screening.  Although  the  contents  of  this 
package  originally  were  expected  to  be  deter- 
mined by  the  consensus  decisions  of  a  panel  of 
experts,  the  program  that  eventually  was  de- 
signed was  heavily  influenced  by  the  results  of  a 
CEA  of  cancer  screening  submitted  by  one  of  the 
consultants  hired  for  the  project  (168,434). 
Moreover,  the  creation  of  the  Medical  Necessity 
Program  has  introduced  an  approximation  of 
cost-effectiveness  methodology  in  Blue  Cross 
and  Blue  Shield  coverage  decisions,  because  cost 
consciousness  has  encouraged  coverage  deter- 
minations that  are  more  closely  related  to  judg- 
ments made  by  the  national  medical  specialty 
societies. 

Further,  greater  emphasis  on  efficacy  data 
also  has  been  promoted  by  Blue  Cross  and  Blue 
Shield's  sponsorship  of  an  Institute  of  Medicine 
study  of  the  CT  scanner  and  subsequent  recom- 
mendation that  coverage  be  restricted  to  uses 
that  the  Institute  found  to  be  supported  by  cur- 
rent evidence  of  efficacy.  Yet,  the  extent  to 
which  these  changes  indicate  that  more  cover- 
age decisions  will  be  tied  to  efficacy  information 
remains  uncertain.  Adequate  evidence  of  the  ef- 
ficacy and  safety  of  medical  technologies  often 


is  not  available,  and  in  the  absence  of  such  evi- 
dence, technologies  judged  to  be  nonexperi- 
mental  and  generally  accepted  usually  receive 
positive  coverage  recommendations  from  both 
the  plans  and  the  national  associations. 

Currently,  however,  Blue  Cross  and  Blue 
Shield  are  studying  contract  and  coding  changes 
that  ultimately  may  facilitate  the  application  of 
cost  and  efficacy  criteria.  Although  Blue  Cross 
and  Blue  Shield  have  their  own  procedure  code, 
plans  are  mandated  to  use  it  only  when  they 
deal  with  national  account  business  (group  ac- 
counts in  which  some  individuals  live  beyond 
the  boundaries  of  an  individual  plan);  other  use 
is  discretionary  and  many  local  affiliates  choose 
to  use  other  codes  for  their  general  subscribers. 
At  present,  however,  the  national  associations 
are  attempting  to  revise  and  update  the  code,  a 
move  which  may  make  it  more  attractive  to 
member  plans  and  more  useful  for  the  identifi- 
cation of  questions  concerning  coverage  issues 
or  inappropriate  use  of  a  technology.  More- 
over, the  Blue  Cross  and  Blue  Shield  Associa- 
tions are  considering  inserting  new  exclusionary 
language  in  their  national  account  contract  and 
recommending  a  model  contract  exclusion 
clause  to  its  local  affiliates  in  order  to  develop  a 
firmer  legal  basis  for  coverage  decision.21  In  rec- 
ognition of  this  need,  many  plans  have  taken  the 
initiative  in  recent  years  and  developed  more 


21Thus,  the  Blue  Cross  Board  of  Directors  and  the  Blue  Shield 
Board  of  Directors  will  be  asked  to  review  and  approve  the  follow- 
ing policy: 

To  recommend  to  the  Boards  that  each  Plan  be  encouraged  to  re-ex- 
amine its  contracts  to  assure  adequate  identification  of  medical  neces- 
sity and  experimental  exclusions,  and  that  the  Boards  offer  the  follow- 
ing language  to  each  Plan  for  consideration. 

"Medically  necessary  services  or  supplies  provided  by  hospital, 
physician  or  other  provider  to  identify  or  treat  an  illness  or  injury  and 
which,  as  determined  by  the  Plan,  are: 

1)  Consistent  with  the  symptom  or  diagnosis  and  treatment  of  the 
condition,  disease,  ailment  or  injury; 

2)  Appropriate  with  regard  to  standards  of  good  medical  practice; 

3)  Not  primarily  for  the  convenience  of  the  member,  his  physician,  or 
other  provider; 

4)  The  most  appropriate  supply  or  level  of  service  which  can  safely  be 
provided  to  the  member.  When  applied  to  an  inpatient,  it  further 
means  that  the  member's  medical  symptoms  or  condition  require 
that  the  services  or  supplies  cannot  be  safely  provided  to  the  mem- 
ber as  an  outpatient. 

Services  and  supplies  which  are  experimental  or  investigational  in 
nature;  meaning  any  treatment,  procedure,  facility,  equipment,  drugs, 
drug  usage,  devices,  or  supplies  not  recognized  as  accepted  medical 
practice  or  by  the  Plan  and  any  of  such  items  requiring  federal  or  other 
governmental  agency  approval  not  granted  at  the  time  services  were 
rendered." 
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specific  exclusionary  language  for  their  con- 
tracts.22 

A  few  plans  in  California,  Pennsylvania, 
Michigan,  Massachusetts,  and  New  York  also 
appear  to  have  taken  the  initiative  in  developing 
a  somewhat  more  systematic  coverage  mecha- 
nism. Perhaps  the  most  sophisticated  decision- 
making process  has  been  adopted  by  California 
Blue  Shield,  which  addresses  a  series  of  ques- 
tions before  making  a  coverage  determination: 

1.  Is  the  procedure  experimental  or  investi- 
gational? 

2.  Is  it  generally  accepted? 

3.  What  is  its  relative  safety? 

4.  What  does  it  cost? 

5.  Is  there  a  procedure  which  costs  less  and 
achieves  the  same  result? 

6.  Is  it  reproducible,  i.e,  can  it  be  used  by 
someone  other  than  the  original  creator? 

7.  Is  it  generally  available? 

8.  Does  it  make  a  difference  in  the  manage- 
ment of  patient  outcomes  or  does  it  just 
serve  an  academic  purpose? 

Although  stage  of  development  and  general  ac- 
ceptance remain  the  most  important  criteria, 
California  Blue  Shield  has  succeeded  in  formally 


"For  example,  Washington,  D.C.,  Blue  Cross/Blue  Shield 
issued  the  following  statement  in  1979  (59): 

Group  Hospitalization,  Inc.,  and  Medical  Services  of  D.C.  have 
always  been  denied  claims  for  hospital  and  medical  services  or  supplies 
which  are  not  medically  necessary.  We're  now  adding  a  contractual 
provision  to  all  non-group  contracts  which  specifically  excludes  serv- 
ices or  supplies  which  are  not,  in  the  judgment  of  the  corporation,  med- 
ically necessary  for  the  diagnosis  or  treatment  of  an  illness. 


placing  the  concepts  of  cost  and  relative  efficacy 
on  its  coverage  agenda.  In  a  few  cases,  where  it 
has  denied  routine  payment  for  a  new  procedure 
that  it  has  determined  costs  more  and  achieves 
the  same  result  as  an  existing  technology,  Cali- 
fornia Blue  Shield  has  applied  an  approximation 
of  CEA  to  its  decisionmaking.  Moreover,  ulti- 
mate authority  to  make  coverage  determina- 
tions for  the  plan  lies  with  the  Medical  Policy 
Committee,  a  24-member  group  composed  of 
physicians  and  4  or  5  public  representatives, 
which  meets  in  a  number  of  locations  across  the 
State  and  invites  representatives  of  the  local 
medical  community  and  the  appropriate  special- 
ty societies  to  attend  its  sessions.  At  least  2  or  3 
weeks  prior  to  the  time  the  Medical  Policy  Com- 
mittee meets,  the  medical  advisor  distributes  an 
agenda  book  containing  all  the  information  that 
has  been  collected  about  the  procedures  to  be 
discussed.  As  a  result,  the  Blue  Shield  coverage 
has  achieved  a  high  degree  of  acceptance  within 
the  California  medical  community,  and  the 
meetings  of  the  Medical  Advisory  Committee 
often  serve  as  a  public  forum  for  the  generation 
of  consensus  about  a  procedure.  Thus,  even 
when  a  decision  is  made  not  to  restrict  payment 
for  a  technology,  the  public  discussion  of  its 
merits  may  result  in  discouraging  its  use.  Fur- 
ther, the  high  visibility  of  the  Blue  Shield  cover- 
age process  in  California  has  augmented  the 
State  plan's  ability  to  identify  coverage  issues, 
because  the  developers  of  new  procedures  tend 
to  seek  the  advice  of  the  Medical  Advisory 
Committee  before  requesting  reimbursement 
(165,537). 


POTENTIAL  USE  OF  CEA/CBA  IN  REIMBURSEMENT 
COVERAGE  DECISIONS:  GENERAL  FINDINGS 


Cost-effectiveness  analysis/cost-benefit  anal- 
ysis (CEA/CBA)  is  based  on  the  assumption 
that  resources  are  limited  and  should  be  ration- 
ally allocated.  Because  it  makes  this  assumption 
of  limits  and  because  it  forces  explicit  considera- 
tion of  alternatives  for  achieving  goals,  CEA/ 
CBA  would  appear  to  be  able  to  contribute  im- 
portant information  to  decisionmakers  con- 
cerned with  acting  as  selective  purchasers  of 


health  services.  Yet  its  appeal  for  reimburse- 
ment officials  seems  severely  tempered  by  the 
small  number  of  well-conducted  CEA/CBA 
studies  available,  methodological  uncertainties 
of  the  techniques,  and  health  policymakers'  lim- 
ited experience  with  their  use.  Moreover,  the 
economic  efficiency  value  embodied  in  cost-ef- 
fectiveness information  may  conflict  with  a 
number  of  other  values  prevalent  in  our  health 
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care  system,  including:  1)  the  practitioner's  obli- 
gation to  do  the  most  for  the  patient,  desire  for 
relative  certainty  in  making  a  diagnosis,  and 
need  to  possess  the  freedom  and  flexibility  to  re- 
spond to  the  circumstances  of  the  individual 
medical  problem;  2)  the  patient's  desire  to  re- 
ceive a  full  range  of  medical  care,  regardless  of 
ability  to  pay;  3)  society's  desire  to  encourage 
innovation  in  order  to  ultimately  improve  care; 
and  4)  society's  goals  in  terms  of  equity  and 
other  noneconomic  values. 

On  the  other  hand,  the  methodology  for  effi- 
cacy and  safety  analysis  is  more  widely  accepted 
than  that  for  CEA.23  Efficacy  and  safety  studies 


"For  a  discussion  of  the  conflict  of  medical  economic  efficiency 
with  efficacy,  see  W.B.  Schwartz  and  P.L.  Joskow, "Medical  Ef- 
ficacy Versus  Economic  Efficiency:  A  Conflict  in  Values,"  N.  Eng. 
}.  Med.,  1978  (548). 


form  part  of  a  relatively  more  established  health 
policy  tradition,  and  the  value  of  protecting  pa- 
tients from  risky,  unsafe,  or  unproven  technol- 
ogy is  less  controversial  (465)  than  weighing 
marginal  cost  and  benefit  tradeoffs  and  not  re- 
imbursing for  some  potentially  beneficial  forms 
of  care.  For  this  reason,  an  approximation  of  the 
cost-effectiveness  concept,  in  which  cost-con- 
tainment goals  encourage  stricter  adherence  to 
safety  and  efficacy  data  might  be  more  appro- 
priate and  acceptable  in  reimbursement  cover- 
age decisions  than  would  formal  CEA/CBA. 
One  of  the  problems  with  this  approach,  how- 
ever, is  that  most  technologies  probably  have 
some  efficacy  in  some  circumstances,  and  ra- 
tional allocation  requires  choosing  among  alter- 
native uses  of  technologies  by  considering  mar- 
ginal costs  and  marginal  benefits  and  relative  ef- 
ficacy for  a  specific  set  of  patient  indications. 


COST  EFFECTIVENESS  IN  RATESETTING 


In  the  last  decade,  the  number  of  hospital 
ratesetting  programs  has  increased  rapidly.  The 
central  feature  of  this  regulatory  mechanism  is 
the  negotiation  of  hospital  rates  in  advance  of 
each  operating  year.  Thus,  Hellinger  points  out 
(294): 

The  key  difference  between  prospective  rate 
setting  and  conventional  methods  of  reimburse- 
ment is  that  hospitals  are  not  paid  the  costs  they 
naturally  incur,  nor  are  they  free  to  unilaterally 
adjust  their  charges  to  cover  their  costs  or  their 
own  interpretations  of  their  financial  require- 
ments; rather,  they  are  paid  at  rates  that  are 
determined  by  another  body  and  that  are  set  in 
advance  of,  and  considered  fixed  for,  the  pro- 
spective year. 

Various  ratesetting  mechanisms,  however, 
differ  considerably  in  their  structure  and  opera- 
tions and  in  the  methods  and  unit  of  payment 
they  use  to  determine  rates.  For  example,  many 
programs  have  the  power  to  set  mandatory 
rates,  whereas  others  rely  on  voluntary  com- 
pliance. More  than  20  programs  are  sponsored 
by  Blue  Cross,  while  12  programs  are  funded  by 
State  governments,  either  through  existing  State 
agencies  or  newly  formed  independent  commis- 
sions (154,155,294). 


Concern  with  the  rapidly  rising  cost  of  health 
care  has  encouraged  the  growth  of  rate  review 
mechanisms.  In  fact,  Federal  legislation  has 
directly  contributed  to  their  development.  Both 
the  1972  amendments  to  the  Social  Security  Act 
(Public  Law  92-603)  and  the  1974  National 
Health  Planning  and  Resources  Development 
Act  (Public  Law  93-641)  provide  for  Federal 
funding  of  experiments  in  ratesetting.  More- 
over, escalating  medicaid  expenditures  have 
prompted  an  increasing  number  of  States  to 
consider  establishing  a  rate  review  system. 

Basically,  ratesetting  agencies  address  the 
need  to  promote  cost  containment  by  encourag- 
ing hospitals  to  operate  efficiently.  By  determin- 
ing in  advance  the  amount  of  revenue  that  will 
be  available,  they  reward  hospitals  that  keep  ac- 
tual costs  below  the  established  rates.  Thus, 
Bauer  points  out  (45): 

The  advantages  seemed  obvious;  if  a  hospital 
could  know  its  payment  rate  before  it  rendered 
its  services,  it  would  have  the  highest  possible 
motivation  to  see  that  these  services  were  pro- 
duced in  the  most  efficient  manner,  since  its 
solvency  would  depend  on  keeping  its  spending 
within  the  limits  of  its  anticipated  revenues.  The 
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hospital  would  have  positive  incentives  for  effi- 
ciency as  well,  since  if  it  could  produce  its  serv- 
ice more  cheaply  than  the  predetermined  rate 
had  allowed,  it  could  pocket  the  difference. 

In  effect,  then,  the  intent  of  rate  review  is  to 
promote  more  efficient  behavior  by  forcing  hos- 
pitals to  live  within  a  fixed  budget,  identify  and 
anticipate  the  costs  of  services  and  facilities,  and 
make  explicit  decisions  about  the  allocation  of 
resources. 

The  record  of  ratesetting  agencies  thus  far, 
however,  is  inconclusive  (45,293,294).  One  sig- 
nificant limitation  is  that  the  lack  of  adequate 
data,  performance  standards,  and  methodology 
makes  it  very  difficult  to  set  rates  of  payment 
that  promote  efficiency.  As  Bauer  notes,  "The 
central  issue  is  how  to  set  rates  in  a  manner  that 
will  neither  underpay  nor  overpay,  but  will  en- 
courage each  institution  to  increase  the  efficien- 
cy with  which  its  services  are  provided"  (45). 
Still  another  major  problem  is  that  ratesetting 


programs  tend  to  focus  on  hospital  operating 
costs  and  do  not  concentrate  on  such  other  im- 
portant areas  as  the  costs  generated  by  physi- 
cians. 

Thus,  it  appears  that  use  of  CEA/CBA  in  rate 
review  systems  will  have  to  await  the  develop- 
ment of  more  sophisticated  ratesetting  methods, 
more  adequate  data,  and  performance  stand- 
ards, and  closer  liaison  with  other  organiza- 
tions, such  as  health  systems  agencies.  It  may  be 
that  analysis  concentrating  on  the  net-cost  end 
of  the  spectrum  of  analysis  (e.g.,  cost  per  unit  of 
service)  is  more  appropriate  for  ratesetting.  If 
so,  ratesetting  may  be  an  area  where  increased 
numbers  of  CEA/CBA-like  studies  could  be 
used.  An  interesting  counterargument  is  that 
ratesetting  could  be  based  on  the  results  of  soci- 
ety-based CEA/CBAs  and  thereby  would  be  less 
oriented  to  a  narrower  efficiency  base  and  more 
toward  social  effectiveness. 
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6. 

Professional  Standards 
Review  Organizations 

INTRODUCTION  AND  BACKGROUND 


As  noted  in  chapter  5,  the  Social  Security  Act 
authorizes  the  medicare  program  to  pay  only 
for  services  that  are  "reasonable  and  necessary," 
and  these  criteria  have  been  generally  regarded 
as  met  when  a  new  technology  is  perceived  to 
have  moved  beyond  experimental  status  toward 
full  clinical  application  and  acceptance  by  the 
local  medical  community.  The  current  policy  of 
the  Department  of  Health  and  Human  Services 
(DHHS)  is  to  use  "broad  strokes  to  sketch  the 
boundaries  of  accepted  good  medical  practice, 
and  (to  leave)  the  fine-tuning  of  the  system  to 
the  'back-end'  mechanism,  the  Professional 
Standards  Review  Organizations  (PSROs)" 
(98). 

The  PSRO  program  is  one  of  the  principal  ex- 
pressions of  Federal  policy  concerning  the  use  of 
medical  services.  The  PSRO  program,  estab- 
lished in  1972  by  Public  Law  92-603,  is  adminis- 
tered by  the  Health  Standards  and  Quality  Bu- 
reau (HSQB),  formerly  the  Bureau  of  Quality 
Assurance,  of  the  Health  Care  Financing  Ad- 
ministration. The  purposes  of  the  program  are 
to  help  improve  the  quality  and  control  the  costs 
of  medical  services  reimbursed  through  Federal 
payment  programs.  The  program  operates  by 
setting  standards  and  criteria  for  the  desired 
level  and  quality  of  medical  services  and  by 
evaluating  against  these  standards  the  services 
actually  provided.  This  process  is  designed  to 
ensure  that  payment  will  be  made  only  when 
services  are  medically  necessary. 

The  PSRO  program  is  based  on  the  concept 
that  medical  professionals  are  the  most  appro- 
priate individuals  to  evaluate  the  quality  of 
medical  services  and  that  effective  peer  review 
at  the  local  level  is  the  soundest  method  for  en- 
suring the  appropriate  use  of  medical  care  re- 
sources and  facilities.  The  PSRO  program  is 
made  up  of  separate  and  independent  organi- 


zations covering  195  geographic  areas.  Each 
PSRO  must  be  substantially  representative  of  all 
practicing  physicians  in  an  area.  The  PSRO  pro- 
gram is  new  and  is  not  yet  fully  implemented. 
Of  the  203  PSRO  areas  in  March  1977,  only  120 
PSRO  agencies  had  been  funded;  100  were  in 
"conditional"  status;  20  were  in  "planning" 
status.  By  April  1979,  the  areas  had  been  con- 
solidated to  number  195.  Of  these,  182  had 
PSROs  in  "conditional"  status,  and  13  were  be- 
ing planned  or  were  inactive. 

Currently,  PSROs  are  required  to  review  in- 
stitutional care — care  provided  in  hospitals  and 
long-term  care  (nursing  home)  facilities — and 
are  authorized  to  regulate  ambulatory  care. 
PSRO  activities  at  present  are  concentrated  on 
inpatient  care  provided  in  short-stay  hospitals. 
The  PSRO  can  delegate  its  review  responsibil- 
ities to  the  hospitals,  or,  if  a  hospital  is  not 
qualified  or  not  willing  to  conduct  the  reviews, 
the  PSRO  conducts  them  itself.  Review  of  long- 
term  care  can  be  delegated  if  such  care  is  a  "dis- 
tinct part"  of  a  short-stay  hospital. 

PSROs  conduct  three  types  of  review  (42 
CFR,  part  466): 

1.  concurrent  review  of  the  medical  necessity 
and  appropriateness  of  admission  to  and 
continued  stay  in  a  hospital; 

2.  medical  care  evaluation  studies  to  assure 
the  quality  and  improve  the  nature  of  the 
utilization  of  health  care  services;  and 

3.  analyses  of  health  care  practitioner,  insti- 
tutional, and  patient  profiles. 

Medical  care  evaluation  studies  and  profile 
analyses  are  used  to  focus  concurrent  review  on 
problem  areas. 
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As  specified  in  the  statute,  PSROs  review 
services  to  determine  whether: 

•  such  services  and  items  are  or  were  medi- 
cally necessary; 

•  the  quality  of  such  services  meets  profes- 
sionally recognized  standards  of  health 
care;  and 

•  such  services  and  items  proposed  to  be  pro- 
vided in  a  hospital  or  other  health  care  fa- 
cility on  an  inpatient  basis  could,  consistent 
with  the  provision  of  appropriate  medical 
care,  be  effectively  provided  on  an  outpa- 
tient basis  or  more  economically  in  an  in- 
patient health  care  facility  of  a  different 
type. 

The  law  requires  that  PSROs  use  norms,  cri- 
teria, and  standards  in  evaluating  medical  serv- 
ices. This  approach  allows  nonphysicians  to 
perform  many  of  the  reviews  and  also  enhances 
the  objectivity  of  the  review  process.  Standards 
are  developed  by  a  consensus  of  physicians, 
based  on  typical  patterns  of  practice  in  the  area 
and  on  such  regional  or  national  information  as 
may  be  available  and  considered  applicable  by 
the  PSRO. 

In  its  early  stages,  the  PSRO  program  has 
concentrated  on  determining  the  need  for  hospi- 
talization. Now  PSROs  are  beginning  to  move 
beyond  the  question  of  necessity  for  hospitaliza- 
tion to  review  of  surgical  procedures  and  review 
of  ancillary  services. 

PSRO  decisions  on  medical  care  utilization 
and  quality  can  be  enforced  in  several  ways.  Re- 
imbursement for  services  provided  can  be  with- 
held by  medicare  and  medicaid  (medicaid  regu- 
lations are  established  in  each  State  and  vary 


somewhat).  For  serious  and  repeated  violations 
of  PSRO  standards,  a  physician's  right  to  be  re- 
imbursed through  medicare  and  medicaid  can 
be  suspended  or  revoked. 

Each  State  with  three  or  more  PSROs  has  a 
statewide  Professional  Standards  Review  Coun- 
cil. Among  other  duties,  the  statewide  councils 
have  the  responsibility  to  disseminate  informa- 
tion and  data  among  the  PSROs  within  the 
State.  At  the  national  level,  a  National  Profes- 
sional Standards  Review  Council  is  established 
by  law.  This  Council  has  several  functions,  one 
being  to  "provide  for  the  development  and  dis- 
tribution, among  Statewide  Professional  Stand- 
ards Review  Councils  and  Professional  Stand- 
ards Review  Organizations  of  information  and 
data  which  will  assist  such  review  councils  and 
organizations  in  carrying  out  their  duties  and 
functions."  Such  information  is  specified  as  in- 
cluding regional  norms  and  standards.  Local 
PSROs  are  not  required  to  accept  model  stand- 
ards issued  by  the  National  Council.  However, 
the  National  Council  has  authority  to  disap- 
prove local  standards  that  deviate  from  model 
standards  if  the  Council  determines  that  the  dif- 
ferences are  not  medically  justified.  The  Nation- 
al PSRO  Council  has  provided  general  guidance 
and  sample  criteria  sets  developed  by  several 
organizations,  including  the  American  Medical 
Association,  under  contract  with  DHHS.  The 
main  purpose  of  these  contracts  has  been  to  de- 
velop criteria  on  medical  necessity  for  hospitali- 
zation for  different  disease  categories.  HSQB 
hopes  that  technical  assistance  and  norms  and 
standards  will  have  an  important  educational 
effect,  as  well  as  a  direct  effect  on  practice 
through  reimbursement  policy. 


REVIEW  CRITERIA 

Criteria  used  in  evaluating  the  quality  of  med- 
ical care  are  usually  grouped  according  to  struc- 
tural, process,  and  outcome  measures.  Struc- 
tural measures  assess  both  the  availability  of 
medical  care  resources  (e.g.,  facilities,  equip- 
ment, and  health  care  personnel)  and  the  quali- 
tative aspects  of  medical  care  personnel  (e.g., 
extent  of  educational  background,  specialty 


board  certification);  process  measures  assess  the 
appropriateness  of  the  medical  care  that  has 
been  provided;  and  outcome  measures  reflect 
the  effect  of  medical  interventions  on  patient 
health  status.  PSROs,  in  their  reviews,  have  re- 
lied primarily  on  process  measures  and,  to  a 
lesser  extent,  on  outcome  measures. 
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Generally,  research  on  quality  of  care  assur- 
ance and  on  the  application  of  quality  of  medi- 
cal care  techniques  has  been  process  oriented, 
i.e.,  assessing  the  appropriateness  of  medical 
care  as  judged  by  the  medical  profession.  PSRO 
concurrent  review  of  admission  to  and  con- 
tinued stay  in  hospitals  is  also  process  oriented. 
In  practice,  moreover,  review  activities  of 
PSROs  have  been  even  more  narrowly  focused. 
The  cost  concerns  of  the  Federal  reimbursement 
program,  together  with  the  controversial  nature 
and  the  uncertain  state-of-the-art  of  reviewing 
the  quality  of  medical  care,  have  limited  reviews 
to  assessment  of  1)  whether  a  diagnosed  prob- 
lem justified  hospital  admission,  and  2)  whether 
continued  hospitalization  was  justified  after  a 
specified  number  of  days  of  inpatient  care. 

Recently,  some  PSROs  have  tried  out  slightly 
more  sophisticated  reviews  than  those  based  on 
length-of-stay  indicators.  Beginning  in  1977,  the 


Delmarva  Foundation  for  Medical  Care  of  the 
eastern  shore  of  Maryland  began  to  use  a  tech- 
nique called  "care  level  and  timeliness  review" 
(CLTR)  to  help  hospitals  find  and  eliminate 
medically  unnecessary  hospital  days  (358).  By 
retrospective  audit,  the  Delmarva  Foundation 
has  evaluated  both  the  timeliness  of  services  and 
the  level  of  medical  care  supplied.  As  alterna- 
tives to  weekend  and  night  admissions  through 
the  emergency  room  (which  contribute  to  un- 
necessary hospitalization)  and  to  lack  of  full 
laboratory  services  on  weekends  (which  can 
lead  to  extra  days  of  hospitalization  until  tests 
have  been  completed),  the  Foundation  examined 
"swing  beds"  with  additional  staffing  for  obser- 
vation of  patients  to  determine  whether  or  not 
they  should  be  admitted,  and  expansion  of  lab- 
oratory services  from  6  to  7  days.  In  both  cases, 
the  reduction  of  unnecessary  days  was  not 
enough  to  justify  the  extra  costs. 


COST-EFFECTIVENESS  APPLICATIONS 


In  exploring  the  actual  and  potential  appli- 
cability of  cost-effective  analysis/cost-benefit 
analysis  (CEA/CBA)  in  the  review  activities  of 
PSROs,  it  is  necessary  to  delineate  several  sepa- 
rate concepts.  First,  PSROs  do  not  normally 
take  a  specific  medical  problem  or  diagnosis  and 
determine  which  of  several  effective  treatment 
approaches  is  the  least  costly.  Instead,  they 
focus  on  reducing  or  eliminating  unnecessary  in- 
patient care.  For  a  given  diagnosis,  PSROs  per- 
form their  review  with  reference  to  standards 
based  on  medical  opinion  that  hospitalization 
for  that  diagnosis  is  or  is  not  needed.  They  also 
review  against  standards  of  how  many  addition- 
al days  of  hospitalization  are  needed.  The  PSRO 
evaluation  does  not,  systematically  at  least,  ex- 
tend to  a  further  determination  that  days  of  hos- 
pitalization beyond  the  standard  (and  the  medi- 
cal care  provided  during  this  period)  are  not 
needed  at  all  or  could  be  just  as  effectively  pro- 
vided through  outpatient  visits.  With  the  explic- 
it goal  of  current  PSRO  reviews  being  to  mini- 
mize unnecessary  hospitalization — not  to  deter- 
mine whether  hospitalization,  outpatient  care, 
or  no  medical  care  at  all  is  most  effective  for  a 


given  medical  problem — PSROs  are  focused  on 
reducing  medical  care  that  is  of  almost  no  value 
to  the  patient,  but  for  which  reimbursement  is 
nevertheless  being  provided. 

Second,  although  outcome  measures,  or  the 
effect  of  medical  interventions  on  patient  health 
status,  are  often  viewed  as  the  best  criteria  for 
evaluating  the  quality  of  medical  care,  these 
have  been  the  most  difficult  criteria  to  develop. 
Hence  the  reliance  on  process,  or  medical  opin- 
ion, criteria.  For  most  medical  problems,  how- 
ever, alternative  avenues  of  treatment — not 
clear-cut,  single  choices — have  been  the  rule. 
Furthermore,  the  effectiveness  of  even  estab- 
lished treatments  is  continually  under  question. 
Taken  together,  the  use  of  several  methods  of 
treatment  for  specific  medical  problems  and  the 
often  uncertain  correlations  between  specific 
medical  interventions  and  the  patient's  health 
status  make  the  development  of  additional  crite- 
ria to  decide  which  interventions  should  or 
should  not  be  reimbursed  very  difficult.  Never- 
theless, the  development  of  such  criteria,  es- 
pecially for  many  of  the  common  interventions 
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that  are  subject  to  review  by  PSROs,  is  not  im- 
possible. Because  PSROs  in  effect  concentrate 
on  deviant  practice  by  establishing  ranges  of  ac- 
ceptable behavior,  boundaries  can  be  set  to  al- 
low reasonable  divergence  based  on  the  possi- 
bilities of  individual  patient  response.  In  con- 
cept at  least,  the  ranges  of  acceptable  behavior 
by  institutions  or  physicians  under  review  could 
be  based  on  information  that  incorporates  cost 
or  cost-effectiveness-related  data.  The  ranges 
could  also  be  based  on  a  more,  though  only 
slightly,  common  form  of  data — efficacy  and 
safety. 

These  two  aspects  of  the  use  of  medical  tech- 
nologies— efficacy  and  safety,  and  cost  effec- 
tiveness— together  define  the  appropriate  use  of 
technologies.  The  PSRO  legislation  established 
a  framework  by  which  appropriate  use  of  medi- 
cal technologies  could  be  evaluated  by  physi- 
cians acting  in  organized  groups  rather  than  as 
individuals.  PSRO's  decisions,  however,  are 
still  based  largely  on  traditional  sources  of  in- 
formation, so  customary  practice  patterns, 
whether  appropriate  or  not,  become  accepted  as 
standard.  The  lack  of  scientifically  derived  in- 
formation on  indications  for  use  and,  especially, 
on  the  cost  effectiveness  of  various  technologi- 
cal applications  hampers  the  development  of  ap- 
propriate standards.  Provided  with  such  infor- 
mation, PSROs  could  perhaps  become  a  more 
effective  mechanism  for  evaluating  medical 
care.  In  its  absence,  PSROs  are  developing  local 
standards  for  medical  services  based  primarily 
on  prevailing  patterns  of  medical  practice,  with 
little  comparison  of  costs  to  benefits. 

On  the  other  hand,  cost-effectiveness  ap- 
proaches have  been  used  in  managing  some 
PSROs  and  in  evaluations  of  the  national  PSRO 
program  itself.  Even  with  the  current  focus  on 
reducing  medically  unnecessary  inpatient  days 
in  short-stay  hospitals,  cost-effectiveness  tech- 
niques can  be  used  for  PSRO  management  pur- 
poses. For  example,  a  PSRO  might  use  analyti- 
cal techniques  to  decide  which  of  a  number  of 
diagnoses  to  focus  on  to  reduce,  by  a  targeted 
number,  the  medically  unnecessary  days  in  a 
given  year. 

Incentives  to  use  cost-effectiveness  ap- 
proaches to  reduce  medically  unnecessary  bed 


days  already  exist  and  have  influenced  specific 
PSROs  in  choosing  among  problem  areas.  The 
PSROs'  administrative  budgets  are  funded  by 
annual  Federal  grants,  and  their  hospital  review 
activities  for  medicare,  medicaid,  and  the  ma- 
ternal and  child  health  and  crippled  children 
programs'  patients  are  paid  from  the  medicare 
trust  funds  (Public  Law  94-182).  Congress  set  a 
limit  on  the  use  of  fiscal  year  1979  medicare 
trust  funds  for  such  hospital  reviews  (44  F.R. 
26770,  May  7, 1979). 

Each  PSRO  is  evaluated  annually  with  regard 
to  its  impact  on  reducing  medically  unnecessary 
hospitalization  and  must  negotiate  formal  annu- 
al "objectives"  with  the  Federal  Government. 
Objectives  are  stated  in  terms  of  "impact"  (e.g., 
"reduce  the  incidence  of  medically  unnecessary 
days  in  PSRO  hospitals  to  5  percent  or  less  as 
measured  by  CLTR"). 

Two  Maryland  PSROs,  the  Delmarva  Foun- 
dation for  Medical  Care  and  the  Baltimore  City 
PSRO,  illustrate  the  effect  of  budget  constraints 
on  seeking  alternative  approaches  to  reducing 
unnecessary  hospitalization  (358).  In  late  1977, 
the  Delmarva  Foundation  for  Medical  Care  de- 
cided that  its  own  collection  of  hospital  data  did 
not  enhance  its  capability  to  reduce  unnecessary 
hospitalization  or  assist  materially  in  iden- 
tifying problem  areas.  By  deciding  to  eliminate 
data  collection  as  a  PSRO  activity  and  accepting 
discharge  abstracts  prepared  by  the  hospitals, 
this  PSRO  was  able  to  reduce  the  concurrent  re- 
view cost  per  discharge  from  $6.50  to  $5.00. 

The  Delmarva  Foundation  also  changed  its 
concurrent  review  process  in  order  to  increase 
its  efficiency.  Nonessential  tasks  such  as  coding, 
norm  assignment,  and  abstracting  are  no  longer 
performed  by  reviewers.  Review  procedures 
have  been  simplified  to  concentrate  only  on  the 
clinical  indications  for  the  need  for  continued 
hospitalization.  A  patient  receiving  intravenous 
therapy,  for  example,  is  automatically  consid- 
ered to  be  appropriately  placed  in  the  hospital. 
By  focusing  only  every  other  day  on  the  clinical 
indications  for  hospitalization,  as  opposed  to 
assigning  a  normative  length  of  stay  in  advance 
for  a  particular  diagnosis,  this  PSRO  dramati- 
cally shortened  the  amount  of  time  required  for 
review  per  patient.  As  a  result,  the  concurrent 
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review  cost  per  discharge  dropped  from  $5.00  to 
$3.25,  yet  the  PSRO's  capacity  to  reduce  unnec- 
essary hospitalization,  as  measured  by  CLTR, 
has  not  been  affected.  OTA  has  no  information 
on  whether  the  quality  of  the  data  itself  was  af- 
fected. 

In  1979,  in  an  attempt  to  cut  costs  further, 
Delmarva  began  exempting  hospitals  with  low 
levels  of  inappropriate  use  from  admission  re- 
view, continued  stay  review,  or  total  review. 
Consequently,  concurrent  review  costs  have 
dropped  below  $2.50  per  discharge.  The  impact 
of  this  action  on  appropriate  hospitalization  has 
not  yet  been  evaluated. 

Recently,  the  Delmarva  Foundation  has  at- 
tempted to  improve  its  utilization  review  proc- 
ess by  employing,  on  a  test  basis,  the  appropri- 
ateness evaluation  protocol  (AEP),  a  recently 
developed  technique  that  compares  care  deliv- 
ered with  an  objective  ideal.  Like  CLTR,  AEP 
measures  unnecessary  days  of  care  created  by 
organizational  problems  and  clinical  judgments 
that  are  grossly  out  of  tune  with  mainstream 
practices.  AEP  goes  one  step  beyond  CLTR, 
however,  because  it  questions  clinical  practices 
that  are  in  local  general  use.  In  addition  to  pro- 
viding more  timely  and  complete  information 
than  CLTR,  AEP  promises  to  be  less  costly. 

This  same  PSRO  has  begun  using  cost-effec- 
tiveness-type analyses  to  identify  unnecessary 
days  of  care  and  to  change  modes  of  physician 
practice  on  a  diagosis-specific  basis.  In  selecting 
topics  for  evaluation,  Delmarva  focuses  on  find- 
ing relatively  low-cost  opportunities  for  making 
substantial  impacts.  A  model  developed  by  the 
PSRO's  statistician  is  used  to  select  topics  for 
regional  medical  care  evaluations  that  will  have 
the  most  impact.  This  model  not  only  identifies 
differences  in  utilization  between  the  PSRO  area 
and  regional  norms,  but  also  identifies  dif- 
ferences within  the  PSRO  area  in  utilization  that 
can  be  changed  through  local  educational  pro- 
grams. 

The  Baltimore  City  PSRO,  partially  in  re- 
sponse to  the  cap  on  review  costs,  has  taken  a 
different  approach  to  maximizing  its  impact 
under  financial  constraints.  Specifically,  it  has 
used  its  data  base  and  computer  capabilities  to 


rank  hospital  and  diagnoses  according  to  excess 
utilization  rates,  thereby  enabling  the  PSRO  to 
focus  its  attention  on  those  hospitals  and  diag- 
noses that  produce  the  highest  incidence  of  un- 
necessary hospital  days. 

Excessive  hospital  days  may  not  be  true  in- 
dicators of  excessive  costs.  Nevertheless,  it  has 
been  the  experience  of  the  Baltimore  PSRO  that 
hospitals  which  have  problems  with  diagnosis- 
specific  lengths-of-stay  also  tend  to  have  prob- 
lems with  high  costs  and  higher  than  average 
death  rates.  Given  these  facts  and  the  PSRO's 
limited  budget,  the  Baltimore  organization  be- 
lieves that  length-of-stay  analysis  is  an  appro- 
priate way  to  identify  inappropriate  and  unnec- 
essary medical  care.  By  identifying  problem 
areas  under  the  length-of-stay  analyses,  the 
PSRO  is  able  to  focus  its  concurrent  review, 
medical  care  evaluation,  and  profile  analysis  ef- 
forts on  those  hospitals  and  diagnoses  where  the 
impact  can  be  greatest. 

This  technique  seems  to  have  worked  for  the 
Baltimore  PSRO.  In  the  first  study  of  this  type, 
that  PSRO  undertook  a  very  careful  review  of 
the  hospital  that  ranked  first  in  excess  utiliza- 
tion. Review  of  the  hospital's  data  indicated 
potentially  serious  problems  with  several  spe- 
cific diagnostic  and  cost  levels.  The  hospital  has 
since  confirmed  the  existence  of  these  problems 
and  corrected  the  excess  utilization  problem. 

Evaluations  of  the  national  PSRO  program 
have  been  conducted  by  the  General  Accounting 
Office  (GAO)  (243),  the  Congressional  Budget 
Office  (CBO)  (121),  and  DHHS  (285,288).  Cur- 
rently, GAO  is  conducting  a  general  review  of 
PSROs,  with  the  intent  of  focusing  future 
studies  on  more  specific  components  of  the  pro- 
gram (404).  CBO  is  also  currently  updating  its 
evaluation  (122),  as  is  DHHS  (138). 

The  GAO  report  was  a  general  review  of  the 
progress  and  problems  in  establishing  PSROs. 
The  CBO  report  more  specifically  addressed  the 
effect  of  PSROs  on  health  care  costs  and  wheth- 
er the  estimated  savings  (if  any)  derived  from 
the  program  outweighed  the  program  costs. 

DHHS's  1977  study  (288)  found  no  reduction 
in  unnecessary  hospital  days  was  achieved 
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through  PSRO  activities.  Its  1978  medicare  rate 
study  (285)  concluded  that  there  was  a  1.5-per- 
cent utilization  reduction,  leading  to  an  esti- 
mated savings  of  $50.5  million.  With  medicare 
review  costs  of  $45.9  million,  the  study  con- 
cluded, there  was  a  net  benefit  of  $4.6  million  (a 
savings-to-cost  ratio  of  1.1  to  1)  or  10  percent  of 
review  costs. 

CBO  analyzed  the  same  data  used  by  DHHS, 
but  reached  a  different  conclusion.  CBO  ad- 
dressed the  effectiveness  of  PSROs  by  posing  the 
following  three  questions: 

1.  How  effective  is  the  program  in  reducing 
hospital  utilization? 

2.  Are  the  savings  associated  with  the  pro-  " 
gram  large  enough  to  justify  the  costs  of 
the  program  itself? 

3.  Are  the  program's  net-savings  large 
enough  to  warrant  the  expectation  that 
PSROs  will  play  a  major  role  in  contain- 
ing health  care  costs? 

CBO  concluded  that  PSROs  have  brought 
about  a  decrease  in  days  of  care  of  roughly  2 
percent  for  medicare  beneficiaries  in  short-stay 
hospitals.  Its  analysis  of  the  DHHS  data  led 
CBO  to  the  conclusion  that  utilization  savings 
would  be  about  30  percent  less  than  review 
costs,  in  contrast  to  DHHS's  conclusion  that 
savings  would  be  10  percent  greater  then  review 
costs.  (Both  estimates,  however,  were  subject  to 
wide  margins  of  error.)  CBO  further  concluded 


that,  even  if  the  10-percent  net  savings  were  ac- 
cepted, those  savings  would  still  be  extremely 
small  relative  to  Federal  expenditures  for  acute 
patient  care — amounting  to  less  than  0.1  percent 
of  medicare  reimbursements  for  inpatient  care 
in  short-stay  general  hospitals. 

One  problematic  issue  is  the  opposite  conclu- 
sions reached  by  CBO  and  DHHS  after  analyz- 
ing the  same  data,  i.e.,  utilization  savings  30 
percent  less  than  review  costs  versus  savings  10 
percent  greater  than  review  costs.  The  CBO 
report  lists  a  number  of  methodological  prob- 
lems that  could  shift  the  conclusion  of  the  cost 
effectiveness  of  the  PSRO  program  either  to  a 
net  loss  or  to  a  net  savings  position,  and  the  per- 
suasiveness of  either  conclusion  might  well  rest 
in  the  eye  of  the  beholder.  Perhaps  more  impor- 
tantly, however,  these  analyses  do  not  examine 
costs  in  relation  to  changes  in  health  outcomes 
that  may  result  from  PSRO  reviews.  Interesting- 
ly, these  evaluations,  which  sometimes  show  a 
small  cost  savings  and  sometimes  show  a  small 
net  cost  increase,  are  critiqued  in  terms  of  the 
value  of  the  national  PSRO  effect.  Implicit  in 
such  a  criterion  is  a  view  of  PSROs  as  a  cost- 
containment  mechanism.  Yet,  if  this  is  indeed 
the  rationale  of  the  PSRO  program,  what  does  it 
matter  if  the  program  does  cost  slightly  less  than 
the  amount  it  saves?  That  net  saving  is  still  in- 
finitesimal compared  to  the  total  cost  of  the  pro- 
grams that  the  PSROs  are  supposed  to  be  con- 
straining. 


USE  OF  CEA/CBA  BY  PSROs:  GENERAL  FINDINGS 


The  PSRO  program  has  the  broad  responsi- 
bility of  seeing  that  Federal  funds  are  used  for 
health  care  services  and  items  that  are  medically 
necessary,  meet  professionally  recognized 
standards  of  care,  and  are  provided  at  the  most 
economical  level  possible  consistent  with  quali- 
ty care.  In  practice,  however,  the  program  has 
concentrated  on  weeding  out  wasteful  care. 
Even  in  this  limited  approach,  cost-effectiveness 
approaches  (loosely,  and  not  formally,  defined) 
have  potential  applicability  in:  1)  setting  stand- 
ards of  medical  care  against  which  actual  prac- 
tices are  judged,  2)  the  internal  management  of 


individual  PSROs,  and  3)  evaluations  of  the  na- 
tional PSRO  effort. 

It  is  theoretically  possible  that  standards 
based  on  CEA/CBA  or  on  other  analyses  incor- 
porating consideration  of  costs  in  relation  to  ef- 
fectiveness could  be  developed  at  the  national  or 
regional  level  and  adapted  for  local  use  by 
PSROs.  Development  of  such  criteria  will  be 
difficult,  and  acceptance  by  PSROs  may  not  be 
gained  easily.  Criteria  for  identifying  quality 
medical  care  have  gone  beyond  gross  measures 
such  as  length-of-stay  norms  to  (still  modest) 


Ch.  6 — Professional  Standards  Review  Organizations  •  67 


criteria  such  as  the  timeliness  of  services  and  the 
level  of  medical  care  supplied  (i.e.,  CLTR)  and 
objective  standards  such  as  the  AEP.  If  usable 
and  relevant  data  on  the  appropriate  uses  of 


various  medical  technologies  are  developed  and 
made  available  to  PSROs,  there  may  be  con- 
siderable potential  for  applying  CEA/CBA  in 
PSRO  review  activities. 
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7. 

Health  Planning  Programs 


INTRODUCTION  AND  BACKGROUND 


Some  of  the  earliest  known  uses  of  cost-bene- 
fit analysis  (CBA)  were  associated  with  health 
planning.  In  the  middle  of  the  17th  century,  for 
instance,  a  prominent  English  physician,  Rich- 
ard Petty,  favored  society's  investing  more  in 
medicine  because  the  value  of  saved  human  life 
far  exceeded  its  cost  (194).  In  this  country,  a 
similar  argument  was  made  by  Lemuel  Shat- 
tuck,  who  in  his  famous  1850  report  used  CBA 
to  justify  his  proposal  for  sanitary  reforms  in 
Boston  (559).  It  is  interesting  to  note  that  most, 
if  not  all,  of  the  early  applications  of  cost-effec- 
tiveness analysis/cost-benefit  analysis  (CEA/ 
CBA)  to  health  care  were  used  by  the  health 
profession  itself  to  justify  further  investment  in 
the  public's  health.  Unlike  the  main  proponents 
of  CEA/ CBA  today — economists  and  others 
who  argue  that  analytical  techniques  are  needed 
to  help  society  spend  its  health  care  dollars  more 
efficiently — earlier  proponents  and  users  of 
CEA/CBA  were  health  professionals  who  ar- 
gued that  society  should  spend  more  money  on 
its  health  care.  These  professionals  rested  their 
argument  on  a  concept  that  still  dominates  the 
discipline  of  health  planning  today — the  concept 
of  "medical  need." 

"Medical  need"  is  not  well-defined.  In  the  dis- 
cussion that  follows,  the  term  "medical  need"  is 
used  to  refer  to  that  health  benefit  which  an  in- 
dividual or  a  population  could  and  "should"  re- 
ceive from  a  given  health  intervention  or  a  pre- 
vention program.  Thus  understood,  medical 
need  is  subject  to  change,  depending  on  the  state 
of  medical  and  health  knowledge  and  the  avail- 
ability of  resources,  and  it  also  carries  with  it  an 
element  of  equity.  Thus,  whereas  a  heart  trans- 
plant was  not  "needed"  20  years  ago  because  the 
procedure  was  not  possible  at  that  time,  the  de- 
termination as  to  whether  there  is  a  "need" 
today  will  depend  on  the  safety  and  efficacy  of 
the  procedure  (i.e.,  the  state  of  knowledge),  the 
personnel,  equipment,  and  money  available  for 


delivering  it,  and  a  notion  of  whether  it 
"should"  be  available.  This  last  consideration — 
should — is  difficult  to  define  because  society's 
underlying  values  are  so  intertwined  with  it.  A 
95-year-old  man,  for  example,  might  not  qual- 
ify for  a  publicly  funded  heart  transplant  under 
any  circumstances.  In  general,  then,  the  concept 
of  "medical  need"  is  defined  as  some  sort  of  ab- 
solute medical  requirement,  within  some 
bounds  of  reasonableness.  This  concept  is  in- 
consistent with  generally  accepted  economic 
principles  which  are  based  on  relative  values 
such  as  price,  utility,  and  alternative  uses  of  re- 
sources. 

Over  the  years,  whether  the  issue  has  been 
manpower  (353),  public  health  departments 
(175),  hospital  construction,  or  comprehensive 
health  planning,  the  principal  consideration  in 
planning  has  been  "need:"  If  the  planner  could 
show  a  medical  need  for  the  resource  or  the 
service,  the  objective  was  set,  the  question  of 
financial  feasibility  often  being  a  secondary  con- 
sideration. 

When  funds  had  to  be  specifically  appropri- 
ated to  meet  an  identified  "need,"  however,  an 
implicit  cost-benefit  tradeoff  must  have  been 
made.  For  instance,  the  1933  Lee-Jones  Report 
(353)  calling  for  more  physicians  would  have  re- 
quired additional  training  funding;  the  1945 
Emerson  Report  (175)  calling  for  complete  pub- 
lic health  coverage  required  additional  Federal 
funding;  and  the  1946  Hill-Burton  Act  calling 
for  increased  hospital  construction  also  required 
Federal  funding.  Although  there  is  no  evidence 
that  formal  CEA/CBAs  were  done  in  any  of 
these  cases,  funding  decisions  were  made  in  a 
political  arena  which  implicitly  weighed  benefits 
against  costs:  When  it  appeared  that  benefits 
outweighed  costs,  funds  were  more  likely  to  be 
appropriated;  when  the  reverse  seemed  true, 
funding  was  curtailed.  In  the  case  of  traditional 
public  health  measures,  the  Federal  Government 
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increased  funding  from  the  mid-1930's  until  the 
early  1950's,  at  which  point  it  perceived  that  ad- 
ditional costs  (investment)  would  exceed  addi- 
tional benefits  and  therefore  it  curtailed  funding 
even  though  the  original  goals  had  never  been 
met  (532).  Recently,  Federal  funding  for  hospi- 
tal construction  has  been  similarly  curtailed,  as 
has  funding  for  physician  training.  Although 
the  essential  ingredients  for  a  CEA/CBA  were 
present,  formal  economic  analyses  of  these  pro- 
grams were  not  ordinarily  performed. 

Most  health  care  resource  decisions  today  are 
made  in  an  economic  semi  vacuum.  Owing  in 
large  part  to  the  third-party  reimbursement  sys- 
tem and  to  the  relative  ease  with  which  the 


health  care  system  can  obtain  private  capital, 
the  recovery  of  investment  has  been  all  but 
guaranteed.  In  other  words,  the  cost-benefit 
tradeoff  has  been  distorted.  Many  believe  that 
this  distortion  has  led  to  the  situation  in  which 
health  care  costs  have  escalated  disproportion- 
ately to  benefits,  a  situation  that  began  with  the 
advent  of  private  health  insurance  in  the  1930's 
and  1940's  and  became  even  more  acute  with  the 
1965  medicare  and  medicaid  amendments  to  the 
Social  Security  Act.  Adding  to  the  problem  has 
been  the  largely  unrestrained  diffusion  of  tech- 
nological achievements  of  our  time.  Basically, 
the  unwritten  rule  has  been  that  if  a  technolog- 
ical advancement  has  medical  merit  it  should  be 
adopted. 


THE  LAW  AND  CONGRESSIONAL  INTENT 


Recent  legislative  measures  reflect  public  con- 
cern over  the  rising  costs  of  health  care.  Partly 
in  anticipation  of  the  economic  effects  of  the 
medicare  and  medicaid  amendments  and  also  to 
help  rationalize  the  health  care  system,  in  1966 
Congress  passed  both  the  comprehensive  health 
planning  (CHP)  amendments  and  the  regional 
medical  program  (RMP).  By  that  time,  scholars 
including  Roemer  had  already  noted  a  positive 
correlation  between  unneeded  beds  and  unnec- 
essary hospitalization  (521),  and  other  research- 
ers were  beginning  to  suggest  that  some  health 
care  procedures,  especially  surgery,  were  being 
performed  unnecessarily.  An  underlying  prem- 
ise of  the  CHP/ RMP  laws  was  a  belief  that  the 
pluralistic  and  fragmented  health  care  system 
simply  needed  a  catalyst  to  help  its  elements  co- 
operate with  one  another.  Cooperation  among 
these  elements,  it  was  believed,  would  reduce 
duplication  of  services  and  facilities,  and  future 
decisions  would  therefore  be  "cost  effective." 
Planning  and  compliance  under  the  CHP/RMP 
laws  were  strictly  voluntary.  These  laws  envi- 
sioned a  health  planning  system  based  on 
"need:"  If  a  particular  service  or  facility  was 
"needed,"  then  it  must  be  worth  the  cost. 

As  it  became  apparent  that  voluntary  plan- 
ning efforts  were  not  effectively  constraining  the 
health  care  system,  Congress  and  individual 


States  began  searching  for  stronger  measures  to 
contain  costs.  First,  in  1964,  New  York  State  en- 
acted a  certificate-of-need  (CON)  law  that  em- 
powered its  planning  agencies  to  deny  reim- 
bursement to  hospitals  for  large  capital  expendi- 
tures unless  the  agency  found  that  there  was  a 
"need"  for  the  service  to  be  provided.  Subse- 
quently, several  other  States  enacted  similar 
laws.  These  State  laws  were  strengthened  by 
Congress  in  1972  when  it  passed  section  1122  of 
the  Social  Security  Act,  allowing  Federal  funds 
for  capital  expenditures  to  be  withheld  if  large 
capital  projects  were  not  approved  by  State 
planning  agencies. 

In  1974,  Congress  enacted  the  National 
Health  Planning  and  Resource  Development  Act 
(Public  Law  93-641),  which  required  all  States 
to  legislate  CON  laws.  The  main  intent  of  this 
Act  was  to  ensure  that  health  resource  decisions 
were  based  on  cost  as  well  as  need.  Section  1513 
of  the  Act  demonstrates  concern  for  health  sta- 
tus, cost,  and  accessibility,  and  sections  1502 
(national  health  priorities)  and  1532  (CON) 
show  similar  concerns. 

By  1979,  Congress  had  passed  amendments  to 
the  Act  specifying  "cost  effectiveness"  as  one  of 
the  criteria  that  local  health  planning  agencies — 
health  systems  agencies  (HSAs) — must  use  in 
their  review  of  the  appropriateness  of  a  health 
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service.  Specifically,  section  1513(g)  was 
amended  to  read: 

...  In  making  the  appropriateness  review 
...  of  a  health  service,  each  health  systems 
agency  shall  at  least  consider  the  need  for  the 
service,  its  accessibility  and  availability,  finan- 
cial viability,  cost  effectiveness  (italics  added), 
and  the  quality  of  service  provided. 

Consideration  of  "cost  effectiveness"  was  also 
required  of  the  State  planning  agencies — State 
health  planning  and  development  agencies 
(SHPDAs) — in  their  review  of  appropriateness. 

The  use  of  the  term  "cost  effectiveness"  in  the 
1979  amendments  suggests  a  growing  emphasis 
on  a  cost-benefit  type  of  tradeoff.  There  is  no 
evidence  that  Congress  intended  that  HSAs  and 
SHPDAs  do  anything  as  elaborate  as  formal 
CEA/CBAs,  but  there  is  evidence  that  Congress 
wanted  these  planning  agencies  to  consider  dur- 
ing the  planning  process  both  the  effectiveness 
of  the  health  care  system  and  its  cost.  The  1979 
amendment  to  section  1502  (national  health  pri- 
orities), for  example,  specified  the  following  as 
national  priorities: 

The  adoption  of  policies  which  will  (A)  con- 
tain the  rapidly  rising  costs  of  health  care  deliv- 
ery, (B)  insure  more  appropriate  use  of  health 
care  systems,  and  (C)  promote  greater  efficiency 
in  the  health  care  delivery  systems  .  .  .  and  the 
development  and  use  of  cost  saving  technol- 
ogy ..  . 

and 

The  strengthening  of  competitive  forces  in  the 
health  care  services  industry  wherever  competi- 
tion and  consumer  choice  can  constructively 
serve  ...  to  advance  the  purposes  of  quality  as- 
surance, cost-effectiveness,  and  access. 

In  addition,  other  parts  of  the  1979  amendments 
severely  restrict  HSAs'  authority  over  health 
maintenance  organizations  (HMOs),  in  the  ap- 
parent belief  that  HMOs  have  inherent  cost-ef- 
fective incentives,  an  issue  more  fully  consid- 
ered in  chapter  10  of  this  volume. 

There  is  evidence,  then,  that  Congress  intends 
that  the  Department  of  Health  and  Human  Serv- 
ices (DHHS)  consider  both  effects  and  costs  in 
its  health  care  resource  considerations  and  that 
this  intent  has  become  more  explicit  over  the 


years.  The  National  Health  Planning  and  Re- 
source Development  Act  and  amendments  have 
provided  five  vehicles  through  which  health  re- 
source allocation  decisions  can,  theoretically,  be 
affected: 

1.  CON  for  large  capital  expenditures  or  ad- 
ditions of  new  services, 

2.  Federal  funds  review  for  certain  health 
programs, 

3.  appropriateness  review  for  existing  facili- 
ties, 

4.  area  health  services  development  fund  for 
planning  grants  for  "needed"  services,  and 

5.  closure  and  conversion  funding  (title  XVI) 
for  assisting  facilities  to  close  "unneeded" 
facilities. 

Of  the  five,  CON  has  been  the  main  vehicle 
available  to  planning  agencies  for  affecting  re- 
source allocation.  Appropriateness  review  car- 
ries with  it  no  sanctions,1  and  neither  the  area 
health  services  development  fund  nor  closure 
and  conversion  funding  (title  XVI)  has  been 
funded  to  date  (189). 

The  1974  National  Health  Planning  and  Re- 
source Development  Act  contained  several  pro- 
visions intended  to  ensure  that  planning  agen- 
cies would  have  technical  and  analytical  capa- 
bilities as  well  as  assistance  presumably  for 
analyses  such  as  CEA/CBA.  First,  the  1974  law 
authorized  funding  the  HSAs  at  a  significantly 
higher  level  than  the  old  CHP  agencies  and  also 
specified  that  HSA  staff  are  to  have  expertise  in 
administration,  in  the  gathering  and  analysis  of 
data,  in  health  planning,  in  financial  and  eco- 
nomic analysis,  and  in  the  development  and  use 
of  health  resources.  Second,  the  law  provided 
for  the  use  of  consultants.  Third,  it  provided  for 
Regional  Centers  for  Health  Planning  to  provide 
technical  assistance  to  agencies.  Fourth,  it  pro- 
vided for  a  National  Health  Planning  Informa- 
tion Center. 

The  law  makes  it  clear,  however,  that  HSAs 
are  not  supposed  to  gather  data,  a  restriction 
which  is  significant  because  available  data  are 
not  sufficient  to  perform  most  CEA/CBA-type 
studies.  In  addition,  it  should  be  noted  that  HSA 
funding  has  been  appropriated  at  a  significantly 

1  Federal  Register,  Dec.  11,  1979. 
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lower  level  than  was  authorized,  which  in  itself  HSAs  have  only  90  days  for  the  entire  CON 
could  preclude  HSAs'  ability  to  perform  valid  process,  another  factor  that  may  limit  HSAs 
studies.  Finally,  it  should  be  noted  that  since       performance  of  CEA/CBA  studies  is  time. 


REGULATIONS,  GUIDELINES,  AND  ASSISTANCE 


Like  Congress,  DHHS  is  explicit  in  its  desire 
that  health  care  resource  costs  be  balanced 
against  benefits.  In  1979,  the  Secretary  issued 
planning  guidelines  to  State  agencies 
(SHPDAs),  State  Health  Coordinating  Councils 
(SHCCs),  and  local  planning  agencies  (HSAs) 
(287).  Contained  in  these  guidelines  was  a 
message  to  planning  agencies  that  their  assess- 
ments of  the  costs  of  implementing  goals  and 
objectives  should  be  made  in  context  with  the 
expected  effects  of  such  implementation  both  on 
the  health  status  of  the  area  residents  and  on  the 
health  care  system  and  its  costs. 

This  message  is  a  clear  departure  from  previ- 
ous ones,  those  messages,  in  effect,  being  that 
planning  should  be  based  strictly  on  "need"  first 
and  "efficiency"  second.  The  departure  from 
previous  policy  does  not  neglect  the  concept  of 
need  in  the  planning  process;  indeed,  identifying 
and  planning  for  the  health  needs  of  the  local 
population  is  still  an  important  theme  in  the 
new  guidelines.  What  distinguishes  the  new 
guidelines  is  that  cost  containment  is  apparently 
as  dominant  a  theme  as  need  (287). 

Cost  containment  is  one  of  the  fundmental 
purposes  of  the  health  planning  program.  Sec- 
tion 1513(b)(3)  of  Public  Law  93-64  requires 
comparison  of  cost  and  benefits. 

Previously,  the  concept  of  cost  containment  was 
generally  expressed  in  terms  of  eliminating  "un- 
necessary" costs,  referring  to  those  costs  that 
produce  little  or  no  benefit.  The  nuances  of  the 
latest  guidelines  suggest  that  other  costs  are  not 
justified  because  they  produce  insufficient  bene- 
fits. 

To  make  judgments  concerning  the  adequacy 
of  benefits  in  relation  to  costs,  agencies  must 
first  be  able  to  identify  changes  in  health  status 
indicators  that  result  from  health  care  invest- 
ments. The  Secretary  states  (287): 


The  HSA  should  describe  in  its  HSP  (Health 
Systems  Plan)  the  expected  effects  of  health  sys- 
tems goals  upon  the  improvement  in  the  popula- 
tion's health  status  ...  as  measured  by  "Indica- 
tors" (which)  are  quantifiable  measures  chosen 
to  reflect  the  health  status  of  the  population  or 
to  represent  how  well  the  health  system  is  per- 
forming. Direct  indicators  (such  as  infant  mor- 
tality or  disability  days,  and  cost  per  patient 
day,  or  time,  or  distance  from  primary  care) 
measure  the  level  and /or  change  in  community 
health  and  in  health  systems  performance.  Indi- 
rect indicators  (such  as  percent  of  the  area  popu- 
lation with  income  below  the  poverty  level  or 
the  number  of  houses  lacking  adequate  plumb- 
ing) indicate  social  or  environmental  conditions 
which  have  been  attributed  to  affecting  the 
health  of  the  area's  residents. 

Then,  goals  are  assessed  in  terms  of  expected 
payoffs  (287): 

The  development  of  health  systems  goals  is 
based  on  an  assessment  of  the  health  service 
needs  of  the  present  and  projected  population, 
and  on  an  assessment  of  health  services  in  terms 
of  availability,  accessibility,  cost,  continuity, 
acceptability ,  and  quality.  The  HSP  summarizes 
the  HSA's  assessment  of  health  service  alterna- 
tives. The  summary (ies)  should  focus  on  possi- 
ble tradeoffs  between  characteristics,  such  as  ac- 
cessibility and  cost,  which  will  affect  the  selec- 
tion of  goals,  and  priorities  among  selected 
goals. 

The  HSP  summarizes  the  major  alternative  ac- 
tions considered  for  accomplishing  each  objec- 
tive. The  choice  between  alternative  actions 
should  obviously  be  predicated  on  technical, 
political,  and  administrative  feasibility,  as  well 
as  cost  effectiveness  analysis. 

In  discussing  the  importance  of  cost,  the  Sec- 
retary states  (287): 

The  HSA  must  consider  costs  when  it  sets 
goals  and  objectives  in  the  HSP,  as  well  as  when 
it  develops  the  recommended  actions  in  both  the 
HSP  and  AIP  (annual  implementation  plan). 
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Costs  of  goals  and  objectives  also  should  be  de- 
scribed to  the  extent  feasible.  Such  costs  will  be 
derived  from  detailed  estimates  of  the  costs  of 
recommended  actions  in  the  HSP  and  AIP. 

In  addition  to  analyzing  the  costs  of  potential 
goals,  objectives,  or  actions,  the  HSA  should 
also  analyze  and  compare  their  expected  bene- 
fits. The  expected  effect  of  any  objective  or  ac- 
tion upon  the  area's  health  status  and  its  eco- 
nomic effects  as  well  as  any  anticipated  im- 
provement in  health  system  performance  should 
be  specified.  The  benefits  of  each  alternative 
should  be  compared  to  the  costs  of  that  alterna- 
tive. 

Furthermore,  the  Secretary  specifically  refers 
HSAs  to  the  results  of  a  Health  Resources  Ad- 
ministration (HRA)  contract  product  which  pro- 
vides the  conceptual  and  technical  framework 
for  a  formal  CBA  in  health  planning  (287). 2 

HRA  has  funded  numerous  other  contracts  to 
assist  in  developing  a  framework  for  resource 
allocation.  At  least  one  is  a  formal  CEA/CBA- 
type  analysis  (68),  but  most  are  more  financially 
oriented,  providing  guidelines  on  capital  budg- 


2The  paper,  "Consideration  of  Benefits  and  Costs:  A  Conceptual 
Framework,"  was  written  by  D.  A.  Dittman  and  K.  R.  Smith 
(146).  In  it,  the  authors  provide  guidelines  by  which  planners  can 
set  priorities,  choose  between  them,  and  perform  formal  cost- 
benefit  analyses. 


eting  (particularly  lifecycle  cost  analysis,  which 
includes  not  only  direct  capital  financing  but 
also  future  operating  expenditures)  (11,363, 
424).  In  February  1980,  HRA,  through  its  Bu- 
reau of  Health  Facilities,  sponsored  a  conference 
on  health  care  capital  in  which  one  of  the  prin- 
cipal topics  was  the  relationship  between  capital 
and  operating  costs  (286).  It  is  generally  con- 
ceded that  such  comprehensive  costing  is  impor- 
tant for  CEA/CBA-type  studies.  Also  important 
is  estimating  health  impacts,  and  for  that,  HRA 
has  funded  the  development  of  methodologies 
such  as  the  health  status  impact  study  by  Kisch, 
et  al.  (315). 

HRA  appears  to  be  making  a  deliberate  and 
orderly  attempt  to  introduce,  interpret,  and  ex- 
pand the  concept  of  the  balancing  of  costs  and 
benefits  in  health  planning.  At  the  same  time, 
there  appears  to  be  a  justified  realization  at 
HRA  that  the  state-of-the-art  of  CEA/CBA  may 
be  too  immature  for  formal,  routine  use  of  these 
analytic  techniques.  HRA's  emphasis  at  present, 
therefore,  is  on  organizational  development  at 
both  the  State  and  local  agency  level,  on  "need" 
assessment,  and  on  HSP  development  (211). 
Meanwhile,  the  more  sophisticated  elements  of 
CEA/CBA  methodology  (e.g.,  lifecycle  capital 
budgeting  and  health  status  assessment)  are  be- 
ing studied  and  slowly  introduced  and  dissemi- 
nated to  the  agency  level. 


CURRENT  USE  OF  ECONOMIC  ANALYSIS 


The  cautious  approach  that  HRA  has  adopted 
with  respect  to  the  use  of  CEA/CBA-type  tech- 
niques for  planning  is  reflected  in  an  analysis  of 
HSA  activities.  An  OTA-funded  study  found  no 
evidence  that  HSAs  have  used  formal  CEA/ 
CBA  to  assist  in  making  resource  allocation 
decisions  (358). 3  In  this  study,  HSAs  most  likely 
to  employ  formal  economic  analyses  during  the 
planning  and  decisionmaking  process  were  iden- 
tified through  an  intensive  interview  process. 


Cost-Benefit  Analysis 

OTA  found  one  example  of  the  use  of  CBA  by 
an  HSA  (Miami  Valley  HSA,  Dayton,  Ohio)  for 
the  majority  of  its  recommendations  in  the  AIP. 
For  the  1980  plan,  the  Miami  Valley  HSA  in- 
cluded 54  CBAs.  In  general,  the  planning  agen- 
cy's staff  contrasted  the  costs  and  cost  savings  of 
a  given  health  program  with  the  productive  val- 
ue of  the  estimated  years  of  life  saved  by  that 
program,  thus  valuing  life  in  terms  of  lost  wages 
averted  (i.e.,  the  human  capital  method  of  valu- 
ing life4).  The  HSA's  analyses  were  presented  as 


3This  study,  conducted  for  OTA  by  Lewin  and  Associates,  ex- 
amined the  use  of  CEA/CBA  by  HSAs  and  by  PSROs. 


"This  method  of  valuing  life  is  discussed  in  ch.  3  of  Background 
Paper  #!.-  Methodological  Issues  and  Literature  Review,  prepared 
by  OTA  in  conjunction  with  this  assessment. 
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supporting  evidence  for  actions  that  had  already 
been  taken  and  objectives  that  already  had  been 
set.  The  methods  of  the  analyses  were  simple 
and  did  not  require  large  amounts  of  staff  time. 
The  HSA  staff  estimated  that  between  2  and  10 
person-days  were  spent  on  each  analysis,  and 
indicated  that  most  analyses  fell  at  the  low  end 
of  this  range. 

Although  the  HSA's  analysts  had  reviewed 
the  pertinent  literature  for  appropriate  parame- 
ters for  the  analyses,  in  most  cases  the  judgment 
applied  to  these  parameters  was  solely  that  of 
the  analysts.  For  example,  a  local  health  educa- 
tion organization  had  estimated  that  a  health 
education  program  could  save  family  medical 
bills  up  to  $290  per  year;  since  the  HSA  analysts 
intuitively  felt  that  a  savings  of  $290  per  family 
per  year  was  too  high  and  could  not  be  corrobo- 
rated, they  arbitrarily  chose  to  use  in  the  analy- 
sis a  savings  of  only  $50,  which  they  discounted 
at  2.5  percent  over  a  15-year  period.  Even  with 
this  major  modification  of  benefit  structure,  the 
benefit/cost  ratio  in  the  analysis  was  calculated 
to  be  26:1.  On  the  strength  of  this  analysis,  the 
Miami  Valley  HSA  included  in  the  AIP  a  recom- 
mendation to  expand  the  health  education  pro- 
gram to  include  6,000  low-income  families. 

The  supporting  information  for  other  similar 
analyses  in  the  AIP  section  of  the  HSP  was  very 
sparse,  too.  Although  elsewhere  in  the  plan 
document  there  was  a  general  introduction  to 
CEA  and  CBA  concepts,  nowhere  in  the  presen- 
tation of  the  analyses  was  there  a  discussion  of 
the  assumptions  required  by  each  of  the  anal- 
yses or  the  levels  of  uncertainty  associated  with 
them.  It  was  clear  that  the  HSA  staff  had  neither 
the  time,  the  resources,  nor  the  expertise  to 
carry  out  valid  CBA  studies.  (Indeed,  for  many 
of  the  programs  which  they  evaluated,  sufficient 
efficacy  information  is  probably  not  available 
for  anyone  to  perform  good  studies).  Yet,  the 
studies  that  were  done  have  been  incorporated 
into  the  AIP  and  apparently  have  gone  virtually 
unchallenged. 

This  example  is  indicative  of  the  bind  in 
which  the  planning  agencies  find  themselves. 
They  are  encouraged,  and  even  mandated,  to  do 
more  than  they  are  perhaps  capable  of  doing. 
The  skills,  data,  and  funds  that  the  agencies 


need  to  perform  high-quality  CEA/CBA-type 
studies  are  not  available.  It  is  important  to  em- 
phasize here  that  OTA  found  no  indication  that 
the  Miami  Valley  HSA  or  any  other  has  used  or 
intends  to  use  CBA  in  its  resource  allocation 
decisionmaking  process.  CON  deliberations  and 
other  resource  allocation  decisions  are  not  sub- 
jected to  CBA. 

Financial  Impact  Analysis 

Although,  for  resource  allocation  decisions, 
OTA  found  no  evidence  of  the  use  of  CEA/ CBA 
by  HSAs  (in  the  sense  that  health  effects  were 
estimated  and  compared  to  costs),  it  did  find 
evidence  of  the  use  of  related  analyses.  These 
"financial  impact  analyses"  or  "net  cost  studies" 
can  take  one  of  several  forms,  some  of  which 
are  broader  than  others. 

One  financial  impact  analysis,  for  example, 
addressed  the  impact  of  terminating/consolidat- 
ing a  hospital  service.  That  analysis  addressed 
the  following  issues: 

•  the  cost  savings  from  closure  or  consolidation 
of  health  care  services; 

•  the  financial  impact  on  the  hospital's  termi- 
nating/losing the  service  and  the  possibility 
that  the  hospital's  rates  would  have  to  be  in- 
creased to  replace  "needed"  revenue; 

•  the  possibility  that  an  offsetting  service  could 
be  found  to  replace  lost  revenue; 

•  the  impact  on  established  physician  practice 
patterns; 

•  the  impact  on  existing  physical  plants;  and 

•  the  possibility  that  other  facilities  could  be 
used  for  "needed"  services  which  will  no 
longer  be  provided  by  the  facility  facing  the 
closure. 

Note  that  this  analysis,  a  typical  financial  im- 
pact analysis,  does  not  consider  changes  in 
health  status,  although  it  does  consider  both 
public  and  private  costs  (a  point  which  is  dis- 
cussed more  fully  later),  and  is  used  to  measure 
changes  in  accessibility  to  services. 

Another  example  of  a  financial  impact  analy- 
sis was  an  analysis  an  HMO  submitted  with  its 
application  for  a  CON  for  an  open  heart  surgery 
unit.  That  analysis  demonstrated  that  the  HMO 
would  save  money  by  building  and  operating  its 
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own  unit  rather  than  contracting  for  the  service 
as  it  was  then  doing.  In  its  review  of  the  HMO's 
application,  the  HSA  countered  with  its  own 
study  showing  that  net  societal  costs  would  be 
increased  by  the  addition  of  the  HMO's  unit,  be- 
cause there  would  be  excessive  capacity  in  the 
area  and  existing  facilities  would  lose  business. 
Again,  this  finding  may  represent  the  clash  of 
private  versus  public  costs  and  benefits. 

Most  HSAs  undertake  much  more  traditional 
analyses,  generally  attempting  to  determine 


whether  a  given  service  meets  minimum  utiliza- 
tion standards,  standards  which  Kristein  has 
characterized  as  resting  more  on  equity  than  ef- 
ficiency (332).  The  national  planning  guidelines 
or  similar  standards  are  often  used  for  these 
determinations.  The  assumptions  are  that  meet- 
ing these  standards  satisfies  basic  medical 
"needs"  and  guarantees  at  least  a  minimal  level 
of  efficiency,  and  is  therefore  cost  effective.  Sel- 
dom, if  ever,  though,  are  costs  weighed  against 
expected  health  benefits. 


POTENTIAL  FOR  USE 

Discussions  with  leading  health  planners,  ex- 
amination of  the  literature,  and  review  of  the 
findings  of  the  OTA-sponsored  empirical  study 
mentioned  above  (358)  yield  the  impression  that 
capital  budgeting  procedures — specifically,  life- 
cycle  cost  analysis — are  one  aspect  of  decision- 
making in  planning  which  is  receiving  substan- 
tial emphasis.  As  noted  previously,  lifecycle 
cost  analysis  includes  the  notion  of  operating 
costs  throughout  the  expected  life  of  the  invest- 
ment, as  well  as  the  capital  depreciation  sched- 
ule. Analysts  argue  that  the  total  cost  of  a  pur- 
chase includes  the  cost  of  operating  it  as  well  as 
buying  it.  Sophisticated  financial  and  account- 
ing techniques  are  required  to  estimate  deprecia- 
tion schedules,  opportunity  costs,  and  the  costs 
of  secondary  effects  such  as  might  be  caused  by 
displacement  of  personnel  or  by  increased /de- 
creased utilization  in  another  part  of  the  system. 

In  effect,  HRA  and  its  Health  Planning  Cen- 
ters appear  to  be  concentrating  on  the  cost  side 
of  the  ledger,  as  opposed  to  the  effectiveness 
side.  They  appear  to  be  telling  the  HSAs  that  so- 
cietal cost  is  much  more  than  initial  purchase 
price,  and  that  as  far  as  effectiveness  is  con- 
cerned, national,  regional,  or  local  standards 
and  guidelines  will  have  to  suffice  for  now.  This 
appears  to  be  a  reasonable  approach,  although 
even  without  good  efficacy  information,  sec- 
ondary systems'  effects  (e.g.,  lost  revenue  to  the 
provider,  changes  in  reimbursement  rates, 
changes  in  physician  practice  patterns)  could  at 
least  be  described  and  their  magnitude  esti- 


mated, as  was  done  by  one  of  the  HSAs  inter- 
viewed (see  earlier  discussion). 

Considering  these  secondary  effects  raises 
again  the  question  of  private  versus  public  costs 
and  benefits,  and  how  these  variables  should  be 
treated.  The  resolution  of  this  issue  has  enor- 
mous implications  regarding  reimbursement 
rates,  competitive  systems  development,  and 
public  responsibility  vis-a-vis  financial  solvency 
of  nonprofit  as  well  as  public  health  care  institu- 
tions. As  an  example,  if  an  HSA  denies  a  com- 
munity hospital  the  right  to  operate  a  service  on 
the  grounds  that  certain  utilization  standards 
(e.g.,  national  guidelines)  are  not  met,  thus 
causing  the  hospital  to  lose  significant  revenue, 
how  should  cost  changes  be  treated?  First  of  all, 
some  utilization  will  be  shifted  to  other  facili- 
ties, which  will  now  operate  at  a  higher  capacity 
and  thus,  presumably,  more  efficiently.  But  the 
losing  facility  will  find  itself  with  one  less  cost 
center  with  which  to  allocate  overhead  and  will 
have  to  absorb  the  now  unallocated  fixed  costs 
elsewhere.  What  responsibility  does  the  HSA 
(or  society)  have  to  the  losing  facility  with  re- 
spect to  assuring  that  it  recover  its  fixed  costs? 
And  how  should  the  costs  be  treated  in  the  anal- 
ysis? 

Furthermore,  revenue-changing  decisions  in- 
evitably affect  reimbursement  rates.  In  Mary- 
land, for  instance,  the  Rate  Review  Commission 
has  been  at  odds  with  that  State's  HSAs,  be- 
cause the  HSA-approved  projects  lend  legiti- 
macy to  requested  rate  increases  (15).  This  class 
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of  problems  may  always  be  present  as  long  as 
facilities  are  reimbursed  on  a  cost  basis.  Since 
one  of  the  purposes  of  the  HSAs'  activity  is  to 
ensure  efficient  utilization  as  measured  by 
standards  such  as  the  national  guidelines,  the 
question  becomes  whether  the  HSAs  are  actual- 
ly capable  of  ensuring  that  such  standards  are 
met.  The  HSA's  control  of  utilization  is  indirect 
at  best,  and  HSAs  cannot  easily  fine-tune  their 
approach.  For  that  fine-tuning,  Professional 
Standards  Review  Organizations  (PSROs)  mon- 
itor physician  activities  in  the  facilities,  but  even 
they  only  or  primarily  monitor  certain  Federal 
beneficiaries.5  To  help  ensure  that  valid  CEA/ 
CBAs  are  done  and  that  their  results  are  effec-^ 
tively  used,  therefore,  it  would  seem  desirable 
to  coordinate  the  planning  process  (i.e.,  HSAs), 
the  quality  assurance/utilization  review  process 
(i.e.,  PSROs),  and  the  reimbursement /rate  re- 
view process. 

Earlier  it  was  noted  that  the  time  available  to 
an  HSA  staff  for  a  CON  analysis  is  very  short, 
probably  considerably  less  than  1  month.  One 
recommended  method  to  overcome  this  time 
constraint  is  to  require  the  applicant  to  do  the 
analysis  (362).  For  a  renovation  replacement 
project,  for  instance,  the  planning  agency  might 
require  the  institution  to  provide  an  analysis  of 
five  alternatives:  1)  reparation,  2)  partial  re- 
placement, 3)  a  decrease  of  capacity  (or  a  small- 
er project),  4)  an  increase  of  capacity  (or  a  larger 
project),  or  5)  no  change.  This  technique  does 
require  tradeoffs  to  be  considered,  and  it  might 
allow  the  planning  staff  time  to  respond  to  an 
analysis  in  an  appropriate  manner  and  often  to 
anticipate  issues. 

CEA  and  Competition 

The  private  versus  public  cost  controversy 
takes  on  great  significance  when  the  HSA  and 
the  HMO  interact.  In  the  example  cited  earlier, 
the  HMO  contended  that  since  its  costs  would 
be  lowered  by  providing  the  service  itself  rather 
than  by  contracting  for  it,  the  HMO  would  be  at 
a  better  competitive  advantage,  thus  forcing  the 
fee-for-service  sector,  ultimately,  to  lower  its 


5See  ch.  6  of  this  volume  for  further  discussion  of  PSRO  activi- 
ties. 


prices  or  to  face  the  possibility  of  losing  sub- 
scribers. This  argument — what  is  good  for  the 
firm  is  good  for  society — is  the  classic  argument 
for  the  free  market  firm.  The  HSA  countered 
with  its  own  analysis  showing  that  the  addition- 
al service  would  result  in  overcapacity  in  the 
HSA  area  resulting  in  higher  societal  costs — a 
classic  social  welfare  argument. 

Congress  has  apparently  resolved  this  contro- 
versy with  the  passage  of  the  1979  Public  Health 
Service  Act  Amendments.  These  amendments 
contain  a  provision  forbidding  HSAs  to  deny  a 
CON  to  an  HMO  as  long  as  the  HMO  can  effec- 
tively demonstrate  that  the  investment  is  benefi- 
cial to  its  own  subscribers  and  that  it  is  more  ef- 
ficient for  the  HMO  than  other  investments. 
That  provision  is  coupled  with  the  law's  require- 
ment that  HSAs  must  consider  "the  strengthen- 
ing of  competitive  forces  in  the  health  services 
industry  wherever  competition  and  consumer 
choice  can  constructively  serve  ...  to  advance 
the  purposes  of  quality  assurance,  cost  effective- 
ness, and  access"  (15). 

The  new  mandate  contained  in  these  amend- 
ments sets  up  potential  forces  that  run  counter 
to  the  conceptual  basis  for  using  CEA/CBA. 
The  rationale  underlying  CEA/CBA  method- 
ology is  that  the  market  is  not  operating  prop- 
erly. The  role  of  CEA/CBA  in  planning  within 
the  context  of  a  competitive  environment  is  un- 
clear. 

CEA  and  HSA  Orientation 

Other  problems  with  the  use  of  CEA/CBA  in 
health  planning — apart  from  the  conceptual  dif- 
ficulties of  using  CEA/CBA  within  the  construct 
of  a  partially  competitive  system — arise  because 
it  is  unclear  whether  the  HSA  is  (or  should  be) 
an  agent  of  the  community  for  which  it  plans,  or 
an  agent  of  the  larger  society.  That  distinction  is 
important  because  the  orientation  which  an 
analysis  takes  is  very  much  dependent  on  the 
orientation  of  both  the  sponsor  and  the  con- 
sumer of  the  analysis. 

OTA  found  that  not  only  is  there  no  stand- 
ard, generally  accepted  set  of  CEA/CBA  meth- 
ods for  all  technologies,  but  also  that  many  of 
the  parameters  chosen,  as  well  as  how  they  are 
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valued,  depend  on  the  purpose  of  the  analysis 
and  for  whom  it  is  done.  Consequently,  the  re- 
sults of  even  a  technically  superior  CEA/CBA 
by  an  HSA  could  vary  significantly  depending 
on  the  importance  which  the  agency  attached  to 
saving  Federal  tax  dollars  or  to  ensuring  that  its 
local  community  received  all  the  health  care 
which  was  needed,  subject  to  prevailing  guide- 
lines. 

One  HSA,  for  example,  may  set  its  priorities 
in  large  part  on  the  basis  of  total  societal  eco- 
nomic payoff  rather  than  health  "need"  alone. 
Thus,  an  analysis  by  this  HSA  would  include 
not  only  health  benefits,  but  also  economic 
benefits  such  as  decreased  health  care  costs  and 
increased  wages.  This  HSA  may  reject  programs 
such  as  large  hospital  capital  investments  be- 
cause they  seem  to  weigh  too  heavily  on  the  cost 
side  without  sufficient  demonstrable  or  proven 
benefits,  opting  instead  for  programs  such  as 
alcoholism  programs  that  increase  working 
hours.  Another  HSA  may  take  the  attitude  that 
since  its  community  health  care  system  is  not 
operating  under  a  budget,  it  should  consider 
"need"  and  efficiency  in  meeting  that  need 
alone.  The  board  of  that  HSA  may  feel  that  in- 
creasing the  productivity  of  the  local  working 
force  is  not  relevant  to  its  decision  process,  that 
is,  including  productivity  variables  will  bias  the 
selection  process  away  from  programs  for  its 
nonworking  constituents. 

There  is  some  conflicting  evidence  as  to  how 
the  HSAs  do  view  themselves,  but  generally  it  is 
believed  that  the  more  local  orientation  holds. 
HRA  Administrator  Foley  stated  that  despite 
the  law's  clear  mandate  to  control  costs,  "from 
the  perspective  of  HSAs  and  State  agencies,  it  is 
not  clear  that  their  priority  is  on  'reasonable 
costs',"  generally  because  of  the  third-party 
reimbursement  system.  Foley  believes  that  "the 
major  proponent  of  giving  priority  to  the  goal 
of  reasonable  cost  is  the  Governors'  Offices  and 
States'  legislatures"  (211). 

Luft  and  Frisvold,  on  the  basis  of  an  intensive 
review  of  CON  decisions  of  two  CHP  agencies, 
stated  that,  at  least  prior  to  1974,  "need"  was 
the  primary  criterion  on  which  decisions  were 
based,  and  that  an  applicant  could  easily  con- 
vince the  agency  that  the  proposed  service  was 


"needed"  even  in  the  face  of  a  surplus,  as  defined 
by  established  standards  (375).  Thus,  Salkever 
and  Bice  were  able  to  show  that  CON  did  not 
constrain  overall  investment  in  capital  budget- 
ing, although  it  did  alter  the  types  of  invest- 
ments made  (530),  the  latter  point  possibly  pro- 
viding a  clue  to  the  potential  role  of  CEA/CBA. 

Most  of  the  studies  cited  above  analyzed  the 
effects  of  the  CON  process  immediately  follow- 
ing its  introduction.  Recently  published  reports 
based  on  surveys  completed  by  the  American 
Health  Planning  Association,  however,  state 
that  the  CON  process  has  yielded  substantial 
savings  (417).  Although  the  surveys  and  thus 
their  results  have  been  questioned,  this  informa- 
tion indicates  a  continuing  controversy  over  the 
effectiveness  of  CON.  Altman,  in  attempting  to 
describe  incentives  of  planners,  noted  that  the 
constituents  of  the  local  HSA  were  the  local  pro- 
viders and  consumers,  both  of  whom  have  eco- 
nomic incentives  to  increase  health  care  re- 
sources in  their  own  area,  particularly,  since 
such  resources  are  ordinarily  paid  for  on  a  much 
broader  basis;  the  constituents  of  the  State  plan- 
ning agency,  however,  are  the  Governor  and  the 
State  legislature,  whose  economic  incentives  are 
to  constrain  costs  (15).  Consequently,  it  is  not 
unusual  for  State  and  local  officials  to  disagree 
on  health  resource  policy  issues  (100).  Interest- 
ingly, however,  roles  are  sometimes  reversed, 
with  local  agencies  being  surprisingly  cost  con- 
scious— recommending  against  CON — and 
State  agencies  being  surprisingly  insensitive  to 
cost — awarding  CON  over  an  HSA's  objections 
(15,189,499). 

Above  it  was  noted  that  according  to  some 
studies,  CON  has  resulted  in  a  shifting  of  capital 
expenditures,  but  not  in  a  decrease  in  total  cap- 
ital expenditures.  The  purpose  of  CEA/CBA  is 
not  to  constrain  costs,  but  to  assist  in  more  effi- 
cient allocation  of  resources,  and  if  properly  ap- 
plied, the  CEA/CBA  technique  possibly  could 
play  a  valuable  role  in  the  CON  process.  The 
first  attempts  to  evaluate  the  effectiveness  of 
CON  initially  focused  on  its  superficial  objec- 
tives (e.g.,  restraining  increases  in  hospital 
beds).  These  studies  indicated  success.  A  second 
generation  of  more  sophisticated  analyses  as- 
sumed that  the  real  objective  of  CON  was  to  al- 
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locate  or  to  constrain  capital  investment  and 
thus  costs,  and  these  analyses  indicated  failure 
(530).  A  third  generation  of  analyses,  which  has 
not  yet  been  performed  to  OTA's  knowledge, 
could  attempt  to  determine  whether  the  shift  in 
resources  caused  by  CON  activity  was  cost  ef- 
fective. That  is,  is  society  better  off  (more 
healthier,  happier)  now  with,  for  example,  so- 
phisticated diagnostic  technology  than  it  would 
be  if  more  beds  had  been  built?  CEA/CBA  in 
theory  could  shed  light  on  this  issue;  the  possi- 
bility of  its  being  used  in  the  future  for  this  pur- 
pose, therefore,  may  merit  consideration. 

A  key  question  remaining  is  how  should 
CEA/CBA  be  used  in  the  planning  process?  To 
constrain  costs  or  to  obtain  the  best  buy  for  the 
money?  The  evidence  indicates  that  CEA/CBA 
is  most  effective  when  it  is  used  within  a  con- 
strained budget,  that  is,  when  choices  have  to  be 
made.  As  was  previously  noted,  HSAs  currently 
plan  in  an  economic  semivacuum.  Although 
they  are  told  to  consider  costs  when  they  plan 
for  need,  HSAs  plan  for  a  health  system  which 
is  under  no  budget  constraint.  In  some  cases,  for 
instance,  HSAs  are  urged  to  consider  the  "finan- 
cial feasibility"  of  a  project;  the  stamp  of 
approval  (i.e.,  CON),  however,  is  a  ticket  for 
financial  feasibility,  since  once  a  CON  is 
awarded,  reimbursement  (full  cost  recovery)  is 
ordinarily  guaranteed  (100).  This  circular  situa- 
tion had  led  to  calls  for  either  external  regional 
budget  controls  or  for  the  less  stringent  measure 
of  requiring  the  agency  to  evaluate  alternative 
projects  to  accomplish  the  stated  objective 
(375).  The  former  would  require  a  major  re- 
structuring of  the  country's  health  care  financ- 
ing system.6  The  latter,  requiring  alternative 
proposals,  is  therefore  advocated  as  a  more  fea- 
sible solution,  and  indeed,  is  practiced  by  some 
HSAs  interviewed  for  the  OTA  study  (358).  The 
latter  mechanism  by  itself,  however,  is  not  suffi- 
cient as  a  cost-containment  strategy  since  it 
allows  the  possibility  of  the  acceptance  of  an 
unlimited  number  of  objectives.  In  this  case,  a 
CEA/CBA  would  simply  help  the  planner  to  en- 


sure that  each  objective  is  accomplished  in  an  ef- 
ficient manner,  albeit  helping  to  ensure  that 
more  benefit  is  obtained  per  dollar  expended. 
By  itself,  however,  use  of  CEA/CBA  is  not  suf- 
ficient for  containing  overall  health  system 
costs. 

The  resource  allocation  decisionmaking  proc- 
ess of  HSAs  is  basically  a  political  process, 
rather  than  a  technical  one.  The  law  envisions 
this  implicitly  by  requiring  that  the  controlling 
board  be  representative  of  local  constituents. 
Nevertheless,  there  are  some  indications  that,  at 
least  during  the  CON  process — which  is  at  pres- 
ent the  primary  means  of  affecting  the  system — 
the  planners  and  their  analysts  are  becoming 
more  sophisticated  at  the  capital  budgeting 
process.  There  is  little  indication,  however,  that 
HSAs  have  attempted  to  estimate  the  health  im- 
plications of  their  decision  process;  there  failure 
to  do  this  is  understandable  given  the  lack  of 
knowledge  among  the  medical  and  health  com- 
munity about  this.  In  general,  decisions  are 
based  on  either  the  national  guidelines7  or  on 
similar  standards,  both  of  which  are  themselves 
often  based  on  "efficient"  utilization  rates  and 
ordinarily  set  by  consensus  of  experts  without 
information  on  marginal  costs  and  benefits. 

There  does  appear  to  be  a  serious  effort  in 
some  instances  to  compare  the  cost  of  programs 
to  the  cost  of  alternative  programs  which  ac- 
complish the  same  objective  (e.g.,  to  compare 
the  cost  of  a  surgicenter  to  the  alternative  cost  of 
performing  inpatient  surgery).  In  these  in- 
stances, benefits  (or  risks)  are  usually  assumed 
to  be  equal  across  alternatives.  If  that  assump- 
tion is  true,  then  the  study  in  which  it  is  made 
can  legitimately  be  called  a  CEA/CBA,  on  the 
net-cost  end  of  the  analytical  spectrum.  If  not 
true,  however,  the  study  is  simply  a  financial 
impact  analysis.  Most  of  the  studies  that  have 
been  done  appear  to  be  examples  of  financial 
impact  analysis,  because  although  equality  of 
benefits  across  alternatives  is  often  claimed,  in 
fact,  the  analysts  seldom  have  any  hard  data  to 
prove  it. 


6Altman  believes  this  would  further  politicize  the  decisionmak- 
ing process  (15);  if  such  is  the  case,  it  would  not  be  conducive  to 
objective  analysis. 


'See  app.  B. 
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USE  OF  CEA/CBA  IN  HEALTH  PLANNING:  GENERAL  FINDINGS 


The  potential  for  increased  use  of  CEA/CBA 
in  health  planning  hinges  on  several  elements: 

•  the  ability  of  HSA  staffs  to  upgrade  their 
technical  skills  and  fundamental  under- 
standing of  CEA/CBA  methods,  and  to  in- 
crease the  amount  of  time  and  financial  re- 
sources available  to  conduct  analyses; 

•  the  establishment  of  budgetary  bounds 
within  which  investment  decisions  are  com- 
peting with  one  another; 

•  the  extent  to  which  market  forces  operate 
and  are  encouraged; 

•  the  ability  to  measure  the  health  effects  of 
technology  and  the  availability  of  such 
data; 

•  the  availability  of  standard  health  care  uti- 
lization data;  and 

•  the  state-of-the-art  of  the  CEA/CBA  tech- 
niques themselves. 

Meanwhile,  health  planning  agencies  could 
perform  analyses  by  explicitly  listing  or  array- 
ing all  elements  which  are  included  or  will  be  af- 
fected by  an  investment.  At  the  most  basic  level, 
this  would  be  a  descriptive  process:  When  costs 
and  effects  can  be  quantified,  that  would  be 


done;  when  they  can  be  combined,  that  would 
also  be  done.  If  there  is  insufficient  evidence  but 
some  probability  of  a  cost  or  effect  resulting 
from  the  decision,  it  would  be  described  and 
commented  on,  but  not  necessarily  quantified. 

This  approach  would  require  systematic  anal- 
ysis but  would  not  require  as  sophisticated  ana- 
lytical skills.  Since  the  health  planning  decision- 
making process  is  political  in  nature  and  ulti- 
mately rests  on  intangible  factors  anyway,  fine- 
ly tuned  studies  with  valid,  aggregated,  and 
quantified  variables  are  not  essential  for  the 
process  to  be  assisted  by  CEA/CBA  methodolo- 
gy. Moreover,  as  analysts  become  more  familiar 
with  formal  CEA/CBA  techniques,  analyses 
could  evolve  toward  increased  sophistication. 

In  this  manner,  the  intent  of  the  health  plan- 
ning law  might  be  met  without  requiring  an  im- 
mediate herculean  effort  to  upgrade  the  tech- 
nical skills  of  planners  and  their  analysts.  This 
suggested  approach  could  also  assist  in  rational- 
izing the  planning  process  and  provide  the  foun- 
dation for  the  time  when,  and  if,  budget  con- 
trols are  ever  imposed  on  the  health  care  sys- 
tem. 
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8. 

Market  Approval  for  Drugs 
and  Medical  Devices 


INTRODUCTION  AND  BACKGROUND 


The  Federal  Government  has  been  authorized 
to  regulate  various  aspects  of  drugs  in  the 
United  States  since  President  Theodore  Roose- 
velt signed  into  law  the  Pure  Food  and  Drug  Act 
of  1906  (113).  That  law  was  in  part  intended  to 
help  prevent  adulteration  and  misbranding  of 
drug  products.  In  1912,  Congress  enacted  the 
"Sherley  amendment,"  which  prohibited  com- 
panies from  making  false  and  fraudulent  cura- 
tive or  therapeutic  claims  on  the  labels  of  their 
products.  In  1927,  Congress  established  the 
Food  and  Drug  Administration  (FDA). 

The  1938  Food,  Drug,  and  Cosmetic  Act, 
passed  by  Congress  in  response  to  a  tragic  event 
in  which  over  100  people  died  from  ingesting  a 
lethal  drug  product,1  increased  the  Federal  Gov- 
ernment's regulatory  control  over  the  marketing 
of  drugs,  devices,  foods,  and  cosmetics.  In  addi- 
tion to  establishing  several  labeling  require- 
ments for  drugs,  the  1938  Act  prohibited  inter- 
state commercial  shipment  of  new  drugs  until 
they  had  been  adequately  evaluated  by  the  Fed- 
eral Government  to  show  that  they  were  safe 
under  the  conditions  of  use  listed  on  their  label. 
It  also  authorized  FDA  to  remove  from  the  mar- 
ket any  drug  it  could  prove  to  be  unsafe.  The 
Durham-Humphrey  Amendment  of  1951  de- 
fined criteria  and  categories  based  on  levels  of 
drug  safety  for  restricting  a  drug  to  legend  (i.e., 
prescription  only)  status. 

Again  stimulated  by  a  disaster,  this  one  in- 
volving fetal  abnormalities  caused  by  the  drug 
thalidomide,2    Congress    enacted    the  Drug 

"Elixir  of  sulfanilamide  made  with  the  lethal  solvent  diethylene 
glycol. 

2Use  of  the  sedative  thalidomide  by  pregnant  women  in  Europe 
and  Japan  resulted  in  several  cases  of  phocomelia.  a  fetal  abnor- 
mality characterized  by  flipper-like  limbs.  The  drug  was  not  ap- 
proved by  FDA  for  marketing  in  the  United  States. 


Amendments  of  1962  (Kefauver-Harris  amend- 
ments). These  amendments  required  drug  man- 
ufacturers to  provide  "substantial  evidence" 
that  their  products  were  efficacious  as  well  as 
safe.  The  amendments  also  required  drug  man- 
ufacturers to  report  promptly  to  FDA  informa- 
tion concerning  the  safety  and  efficacy  of  their 
marketed  products  and  strengthened  FDA's  au- 
thority to  remove  unsafe  or  ineffective  drugs 
from  the  market.  In  addition,  the  1962  amend- 
ments authorized  the  notice  of  claimed  inves- 
tigation for  a  new  drug  (IND)  process — a  legal 
mechanism  used  by  FDA  to  regulate  human  in- 
vestigations of  drugs. 

The  Federal  Government  was  given  some  au- 
thority to  regulate  medical  devices  in  the 
original  1938  Food,  Drug,  and  Cosmetic  Act. 
Congress  substantially  increased  this  Federal 
regulatory  authority,  however,  by  passing  the 
Medical  Device  Amendments  of  1976  (116).  The 
1976  amendments  authorize  FDA  to  require  de- 
vice manufacturers  to  demonstrate  acceptable 
levels  of  safety  and  effectiveness  for  their  prod- 
ucts. They  establish  three  categories  of  prod- 
ucts, and  each  category  has  a  different  level  of 
Federal  control.  Medical  devices  that  are  im- 
planted, used  in  supporting  or  sustaining  human 
life,  of  substantial  importance  in  preventing  im- 
pairment of  human  health,  or  that  pose  a  poten- 
tial unreasonable  risk  of  illness  or  injury,  for  ex- 
ample, usually  are  subjected  to  premarketing 
evaluation  of  safety  and  efficacy.  The  1976 
amendments  permit  FDA  to  ban  or  remove 
from  interstate  commerce  unsafe  or  ineffective 
devices.  They  also  allow  the  Agency  to  restrict 
the  use  of  a  device  to  persons  with  specific  train- 
ing or  experience  or  to  those  in  specified  facili- 
ties if  the  device's  safety  and  effectiveness  can- 
not be  assured  in  general  use. 


85 
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FDA'S  PRESCRIPTION  DRUG  MARKET  APPROVAL  PROCESS 


Since  1962,  FDA  has  used  its  statutory  and 
regulatory  authority  to  establish  an  extensive 
system  for  evaluating  virtually  every  new  pre- 
scription drug  prior  to  its  release  into  the  gener- 
al medical  marketplace.  By  law,  FDA  must  base 
its  evaluation  of  prescription  drugs  on  two  cri- 
teria: safety  and  efficacy.3  The  procedures  used 
by  FDA  in  this  premarket  approval  process,  de- 
scribed in  detail  elsewhere  (119,144),  are  briefly 
summarized  below.  To  initiate  human  investi- 
gations involving  agents  legally  defined  as  "new 
drugs,"  the  drug  "sponsor"  (e.g.,  a  manufac- 
turer or  an  independent  investigator  of  a  new 
drug)  must  obtain  FDA's  approval.  To  obtain 
such  approval,  the  sponsor  submits  to  FDA's 
Bureau  of  Drugs  (BOD)  an  IND,  describing  the 
qualifications  of  the  investigators  and  the  design 
of  the  planned  trials.  In  the  IND  review  process, 
BOD  also  examines  data  regarding  the  phar- 
macology and  toxicology  of  the  applicant  drug 
collected  in  animal  studies  and  in  human  studies 
that  were  not  subject  to  FDA  approval  (e.g., 
those  conducted  in  foreign  countries).  If  the 
sponsor's  IND  is  approved,  the  sponsor  may 
proceed  with  clinical  testing  (i.e.,  testing  in 
human  subjects)  in  the  United  States.  After  com- 
pleting clinical  testing  under  IND,  the  sponsor 
files  with  BOD  a  new  drug  application  (NDA) 
that  describes  in  detail  the  results  of  IND  clinical 
trials.  The  applicant  drug  has  usually  been 
tested  in  500  to  3,000  human  test  subjects  (306). 
By  filing  an  NDA,  the  sponsor  is  requesting 
FDA's  permission  to  market  the  new  drug  in  in- 
terstate commerce. 

According  to  Dr.  Marion  J.  Finkel,  Associate 
Director  for  New  Drug  Evaluation,  BOD,  FDA, 
the  review  process  for  INDs  and  NDAs  proceeds 
as  follows  (206): 


Each  IND  and  NDA  application  is  reviewed 
by  a  team  of  FDA  scientists:  a  physician,  a  phar- 
macologist, a  chemist,  a  pharmacokineticist, 
usually  a  biometrician,  and  when  applicable,  a 
microbiologist.  Important  NDA's  are  then  pre- 
sented for  consideration  to  advisory  commit- 
tees, of  which  the  Bureau  of  Drugs  has  13,  con- 
sisting of  extramural  [mostly  nongovernmental] 
experts  in,  principally,  the  subspecialities  of 
medicine,  clinical  pharmacology,  and  biomet- 
rics. The  committees  recommend  to  the  FDA 
whether  or  not  an  NDA  should  be  approved  for 
marketing  and,  if  so,  under  what  labeling,  and 
whether  the  sponsor  should  be  requested  to  per- 
form additional  studies  in  the  postmarket- 
ing phase.  When  recommendations  are  made 
against  approval,  the  committees  provide  advice 
on  new  studies  to  be  done  by  the  sponsors  to  ex- 
plore the  drug's  safety  and  effectiveness  further. 
The  use  of  advisory  committees  is  the  FDA's  pri- 
mary method  for  broadening  the  decisionmak- 
ing process. 

During  the  NDA  review  process,  FDA  re- 
viewers analyze  the  sponsor's  summaries  of 
data — and  when  needed  the  actual  raw  data — 
generated  from  clinical  investigations.  A  major 
task  for  BOD  during  this  review  process  is  to  en- 
sure that  clinical  experimental  data  support  the 
sponsor's  claims  for  the  drug's  safety  and  ef- 
ficacy that  appear  in  the  drug's  labeling.  The 
NDA  review  process  usually  entails  delibera- 
tions between  FDA  and  a  drug's  sponsor  regard- 
ing claims  for  safety  and  efficacy.  If  FDA  con- 
cludes that  an  NDA  is  deficient,  it  usually  will 
require  the  sponsor  to  generate  new  data,  modi- 
fy its  NDA  with  additional  information,  and  re- 
submit the  application  to  the  Agency.  Once  a 
sponsor  obtains  an  NDA  approval  from  FDA,  it 
is  authorized  to  market  the  drug  in  interstate 
commerce  for  only  the  specific  indications  (uses) 
that  have  been  authorized  by  FDA. 


3Food,  Drug,  and  Cosmetic  Act  S505,  21  U.S.C.  S  355  (1976) 
part  314  (1979).  The  Act  uses  the  term  "effectiveness"  instead  of 
"efficacy." 
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FDA'S  MEDICAL  DEVICE  EVALUATION  PROCESS 


The  Medical  Device  Amendments  of  1976 
(Public  Law  94-295)  require  FDA's  Bureau  of 
Medical  Devices  (BMD)  to  classify  each  medical 
device — on  the  basis  of  the  level  of  regulation 
necessary  to  ensure  safety  and  efficacy — into 
one  of  three  regulatory  classes.  Products  placed 
in  the  class  I  category — those  requiring  the  least 
controls  to  ensure  their  safety  and  efficacy — are 
subject  only  to  general  controls,  including  pre- 
market  notification,  registration  of  the  man- 
ufacturer, prohibition  of  product  misbranding 
or  adulteration,  adherence  to  FDA-promulgated 
good  manufacturing  practices,  and  compliance 
with  various  recordkeeping  requirements.  Class 
II  products — those  for  which  general  controls 
are  deemed  inadequate  to  ensure  their  safety 
and  efficacy — must  comply  with  performance 
standards  either  established  or  recognized  by 
BMD  when  existing  information  permits  devel- 
opment of  such  standards.  General  controls  also 
apply  to  class  II  devices  unless  superseded  by  a 
standard.  Class  III  devices — those  for  which 
neither  general  controls  nor  performance  stand- 
ards alone  are  sufficient  to  ensure  their  safety 
and  efficacy — are  subject  to  premarketing  ap- 
proval. (BMD  can  also  require  premarketing  ap- 
proval of  devices  for  which  insufficient  informa- 
tion is  available  for  the  development  of  ap- 
plicable performance  standards.)  Class  III  de- 
vices are  subject  to  general  controls. 


The  process  BMD  uses  to  evaluate  the  safety 
and  efficacy  of  new  class  III  medical  devices  is 
similar  to  that  used  by  BOD  in  its  NDA  reviews. 
BMD,  however,  is  required  by  law  to  use  its  ad- 
visory panels  during  a  product's  review, 
whereas  BOD's  use  of  advisory  panels  is  discre- 
tionary. BMD  is  also  authorized  to  use  another 
premarketing  approval  process,  the  product  de- 
velopment protocol  (PDP).  PDP  was  designed 
to  encourage  the  development  of,  and  to  stream- 
line the  approval  process  for,  innovative  medi- 
cal devices.  During  a  PDP  process,  investigation 
of  a  device  and  the  development  of  information 
necessary  for  its  premarket  approval  are  merged 
into  one  regulatory  mechanism.  BMD  works 
closely  with  the  manufacturer  while  informa- 
tion to  support  the  device's  safety  and  effec- 
tiveness is  being  developed.  The  1976  Medical 
Device  Amendments  also  include  an  investiga- 
tional device  exemption  (IDE)  provision.  IDE 
allows  FDA  (technically,  the  Secretary  of 
Health  and  Human  Services)  to  exempt  a  new 
device  from  other  provisions  of  the  amendments 
to  permit  controlled  testing  of  new  devices  prior 
to  commercial  marketing.  The  IDE  process  is 
similar  to  the  IND  process  that  BOD  uses.4 


445  FR3732. 


SAFETY  AND  EFFICACY  CRITERIA 

The  Food,  Drug,  and  Cosmetic  Act  mandates 
FDA  to  require  a  drug  sponsor  to:  1)  collect,  by 
all  methods  reasonably  applicable,  data  that 
demonstrate  a  drug's  safety,  and  2)  generate 
"substantial  evidence"  from  controlled  trials  to 
show  that  the  drug  is  efficacious  for  use  under 
the  conditions  set  forth  in  the  proposed  labeling 
(144).  Although  the  Act  provides  no  definitions 
and  little  guidance  for  the  meanings  of  safe  and 


effective,5  it  does  describe  "substantial  evi- 
dence" of  effectiveness  as  follows  (144): 

5OTA  differentiates  "efficacy"  and  "effectiveness"  as  follows 
(465): 

Efficacy:  The  probability  of  benefit  to  individuals  in  defined  popu- 
lations from  a  medical  technology  applied  for  a  given  medical 
problem  under  ideal  conditions  of  use.  Effectiveness:  Same  as  ef- 
ficacy except  that  it  refers  to  ".  .  .  average  conditions  of  use." 

Although  the  FDA  statutes  use  the  terms  "effective"  and  "effec- 
tiveness," the  definitions  implicit  in  the  statutes  appear  to  mean 
"efficacy."  Thus,  "efficacy"  is  used  in  the  present  report. 
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The  term  "substantial  evidence"  means  evi- 
dence consisting  of  adequate  and  well-controlled 
investigations,  including  clinical  investigations, 
by  experts  qualified  by  scientific  training  and  ex- 
perience to  evaluate  the  effectiveness  of  the  drug 
involved,  on  the  basis  of  which  it  could  fairly 
and  responsibly  be  concluded  by  such  experts 
that  the  drug  will  have  the  effect  it  purports  or  is 
represented  to  have  under  the  conditions  of  use 
prescribed,  recommended,  or  suggested  in  the 
labeling  or  proposed  labeling  thereof. 

Because  no  drug  is  absolutely  safe  or  always 
effective  under  all  conditions  of  use,  FDA 
weighs  a  drug's  benefits  in  comparison  to  its 
risks.  In  FDA's  evaluation  process,  an  applicant 
drug's  safety  and  efficacy  are  compared  to  the 
safety  and  efficacy  of  approved  products  or 
medical  procedures  used  to  achieve  similar  ther- 
apeutic objectives.  Comparisons  are  also  made 
with  the  effects  of  inert  substances  (placebos). 
According  to  Dr.  Marion  J.  Finkel  of  FDA  (204): 

A  drug  cannot  be  considered  safe  if  it  is  less  ef- 
fective and  has  more  side  effects  than  other 
drugs  labeled  for  the  same  indication.  It  might 
be  safe  for  certain  subpopulations  or  certain  re- 
stricted indications,  however. 

According  to  Dr.  Robert  Temple  of  FDA  (594): 

We're  not  supposed  to  refuse  approval  be- 
cause of  lesser  effectiveness,  but  we  can  on 
grounds  of  lesser  safety  which  reduces  the  bene- 
fit/risk ratio.  Relative  effectiveness  can  be  noted 
in  labeling. 

FDA  also  uses  such  comparisons  when  selecting 
NDAs  for  its  priority  review  or  "fast  track"  re- 
view process. 


FDA  has  published  25  clinical  guidelines  that 
define  the  types  of  studies  it  deems  appropriate 
to  use  to  establish  safety  and  efficacy  of  a  drug. 
The  performance  criteria  used  to  evaluate  an  in- 
dividual drug's  safety  and  efficacy,  however, 
can  vary  among  different  pharmacological 
classes.  This  variance  results  from  the  imprecise 
nature  of  pharmacologic  intervention  in  disease. 
The  benefits  and  risks  of  a  drug  can  vary  from 
disease  to  disease,  from  population  to  popula- 
tion, and  from  clinical  situation  to  clinical  situa- 
tion. Drug-induced  hair  loss,  for  example,  is 
often  viewed  as  an  acceptable  risk  of  taking  a 
drug  that  is  an  effective  treatment  for  cancer, 
especially  when  no  other  effective  therapy  ex- 
ists. The  same  side  effect,  however,  would  prob- 
ably not  be  acceptable  for  a  drug  that  reduces 
the  severity  of  a  minor,  self-limiting  condition. 


The  complete  risk-benefit  analysis  of  a  drug, 
therefore,  is  not  based  solely  on  statistical 
evaluation  of  data  on  safety  and  efficacy  gener- 
ated from  clinical  trials  but  also  on  "the  context 
of  the  disease  for  which  [the  drug]  is  intended, 
the  availability  of  other  therapeutic  modalities, 
including  other  forms  of  pharmacologic  thera- 
py, and  public  health  implications  of  its  avail- 
ability" (144). 


The  task  of  evaluating  the  safety  and  efficacy 
of  drugs,  as  well  as  FDA's  ability  to  perform  this 
task,  has  been  subjected  to  extensive  analysis 
and  debate  (119,144,242,613). 


CURRENT  ECONOMIC  CONSIDERATIONS  IN  FDA'S  DRUG  AND 
MEDICAL  DEVICE  APPROVAL  PROCESSES 


FDA  does  not  directly  use  economic  criteria 
in  its  approval  processes  for  drugs  and  medical 
devices.  The  Agency's  primary  statutory  au- 
thority, the  Food,  Drug,  and  Cosmetic  Act,  nei- 
ther authorizes  nor  prohibits  the  use  of  econom- 
ic criteria  in  FDA's  evaluation  of  applicant 
drugs  and  devices.  The  legality  of  using  cost  ef- 
fectiveness to  help  evaluate  new  drugs  and  de- 
vices has  not  been  tested. 


Although  FDA  does  not  formally  assess  the 
potential  economic  impact  a  drug  or  medical  de- 
vice might  have  on  the  allocation  of  health  care 
resources,  some  FDA  actions  may  be  based  indi- 
rectly on — or  taken  in  spite  of — economic  con- 
siderations. Further,  some  FDA  actions  certain- 
ly have  economic  impacts.  Several  examples  are 
cited  below. 
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Classifying  the  Potential  Therapeutic  Impor- 
tance of  New  Drugs:  FDA  sets  priorities  for  its 
review  of  new  drugs  according  to  the  potential 
therapeutic  usefulness  of  each  applicant  drug. 
This  priority-setting  process  supposedly  begins 
within  6  months  after  a  sponsor  submits  an  IND 
to  FDA  (206).  On  the  basis  of  preliminary  in- 
formation about  a  drug's  pharmacological  ef- 
fects and  on  data  from  clinical  trials,  a  drug  is 
assigned  a  number  and  a  letter  derived  from  the 
following  classification  schemes  (221): 

Numerical  Classification 

1  =  New  molecular  entity  not  previously 

marketed  in  the  United  States. 

2  =  New  salt,  ester,  or  derivative  of  an  active 

moiety  marketed  in  the  United  States. 

3  =  New  formulation  of  a  compound  mar- 

keted in  the  United  States. 

4  =  New  combination  of  two  or  more  com- 

ponents not  previously  marketed  togeth- 
er in  the  United  States. 

5=  Product  duplicates  a  drug  marketed  by 
another  firm  in  the  United  States. 

6  =  Product  already  marketed  by  some  firm 
in  the  United  States;  approval  being 
sought  for  new  indication  of  use. 

Letter  Classification 

A  =  major  therapeutic  advance  over  other 

currently  available  drugs,  etc. 
B  =  modest  (or  moderate)  therapeutic  ad- 
vance over  other  currently  available 
drugs,  etc. 

C  =  little  or  no  appreciable  therapeutic  ad- 
vance over  other  currently  available 
drugs,  etc. 

The  purpose  of  using  a  drug  classification 
scheme  of  this  type  is  to  expedite  the  review  of 
data  for  drugs  that  represent  important  new 
therapeutic  entities.  FDA  seeks  to  review  NDAs 
submitted  for  drugs  it  assigned  a  1A  or  IB  rating 
as  expeditiously  as  possible;  such  reviews  re- 
ceive high  priority  by  BOD  (206). 

Even  though  the  criteria  used  to  classify  a  new 
drug  are  based  on  scientific  data  relating  to  a 
drug's  clinical  safety  and  efficacy,  there  are  in- 
direct economic  consequences  of  FDA's  selec- 
tion process.  The  expeditious  review  and  result- 
ant early  marketing  of  a  new  drug  that  repre- 


sents a  major  therapeutic  breakthrough  in  the 
treatment  of  a  heretofore  uncontrollable  disease 
could  help  reduce  the  use  of  existing  treatment 
measures,  such  as  hospitalization  and  surgery. 
The  recently  introduced  drug  cimetidine,  for  ex- 
ample, appears  to  provide  safe,  effective,  and 
relatively  inexpensive  (compared  to  hospitaliza- 
tion or  surgery)  treatment  for  duodenal  ulcers; 
using  cimetidine,  some  ulcer  patients  may  avoid 
hospitalization  (203,632).  FDA  rated  cimetidine 
as  a  1A  drug.  If  indeed  the  use  of  cimetidine 
reduces  ulcer  patients'  hospitalizations  and 
surgery,  then  an  expeditious  FDA  review  and 
approval  of  the  drug  could  help  reduce  medical 
expenditures  for  the  treatment  of  duodenal 
ulcers.  A  complete  analysis  would  include  cal- 
culations of  the  potential  economic  impacts  of 
delaying  the  review  of  one  or  more  other  INDs 
or  NDAs,  in  order  to  expedite  cimetidine's  ap- 
plication review. 

According  to  FDA's  Dr.  Marion  J.  Finkel 
(205): 

NDAs  for  1A  and  IB  drugs  are  full  NDAs, 
containing  all  of  the  safety  and  efficacy  data  re- 
quired for  any  NDA.  Expeditious  review  of 
these  merely  means  that  the  NDAs  do  not  wait 
their  turn  in  the  pipeline  but  are  reviewed  before 
NDAs  with  lesser  classifications  .  .  . 

Rarely,  FDA  will  accept  an  NDA  for  a  1A 
drug  without  as  much  long  term  human  (or  ani- 
mal) safety  data  required  for  NDA  approval. 

In  at  least  some  cases,  however,  the  expedi- 
tious review  of  an  important  new  drug  might  re- 
sult in  an  incomplete  assessment  of  the  drug's 
safety,  and  an  unexpectedly  significant  level  of 
adverse  reactions  to  the  drug  might  occur.  To 
help  prevent  such  an  occurrence,  FDA  can — and 
sometimes  does — ask  a  sponsor  to  conduct  post- 
marketing surveillance  as  a  condition  of  ap- 
proval for  a  new  drug  (131). 

Use  of  Abbreviated  New  Drug  Applications 
(ANDA)  and  "Paper"  NDAs:  Two  examples  of 
the  indirect  use  of  economic  considerations  in 
FDA's  drug  approval  processes  are  the  ANDA 
and  the  so-called  "paper"  NDA.  The  ANDA 
process  enables  a  drug  manufacturer  to  obtain 
from  FDA  marketing  approval  for  a  "generic" 
drug  product  that  is  purported  to  be  an  identical 
version  of  an  already  approved  product,  usually 


66-220  0-80-7 


90  •  The  Implications  of  Cost-Effectiveness  Analysis  of  Medical  Technology 


after  the  originator's  patent  has  expired.  Be- 
cause of  FDA's  earlier  interpretation  of  the 
Food,  Drug,  and  Cosmetic  Act,  ANDAs  are  cur- 
rently only  used  for  products  originally  ap- 
proved by  FDA  between  1938  and  1962.  FDA  is 
preparing  a  policy,  however,  that  will  permit 
the  use  of  ANDAs  for  products  approved  after 
1962.  A  manufacturer  can  obtain  an  AND  A  es- 
sentially by  demonstrating  that  it  complies  with 
FDA's  current  good  manufacturing  practices 
and  labeling  requirements  and  that  it  can  make 
a  product  that  is  at  least  chemically  equivalent 
and  supposedly  bioequivalent  to  the  origina- 
tor's. Although  evidence  of  chemical  equiva- 
lence is  always  required,  the  FDA  Commission; 
er  can  forgo  requiring  evidence  of  bioavailabil- 
ity in  the  ANDA  process  for  a  given  drug. 

"Paper"  NDAs  are  designed  to  permit  a  drug 
manufacturer  to  meet  the  requirements  for  sub- 
mitting evidence  of  the  safety  and  efficacy  of  its 
post-1962  product  by  citing  existing  data  from 
published  clinical  trials  involving  a  chemically 
equivalent,  previously  approved  product.  The 
legality  of  "paper"  NDAs  was  challenged  in 
court  by  the  Pharmaceutical  Manufacturers  As- 
sociation (PMA)  and  a  few  major  drug  manu- 
facturers (473).  The  court  dismissed  the  suit  on 
the  grounds  that  the  plantiffs  had  not  exhausted 
their  administrative  remedies  within  FDA,  so 
PMA  has  filed  a  petition  that:  1)  questions 
FDA's  authority  to  implement  a  "paper"  NDA 
policy,  and  2)  requests  that  if  such  a  policy  is 
implemented  by  FDA,  it  be  done  through  the  of- 
ficial notice  and  comment  rulemaking  pro- 
cedure. 

The  primary  purpose  of  both  the  ANDA  and 
"paper"  NDA  modifications  in  the  new  drug  ap- 
proval process  is  to  facilitate  the  marketing  ap- 
proval of  drug  products  identical  to  those  that 
FDA  has  previously  found  to  be  safe  and  effica- 
cious on  the  basis  of  data  supplied  in  a  product's 
original  NDA.  These  mechanisms  are  designed 
to  prevent  unnecessary  work  burdens  on  FDA's 
resources.  The  use  of  ANDAs  and  "paper" 
NDAs  is  supposed  to  increase  the  availability  of 
FDA  staff  to  review  NDAs  submitted  for  inno- 
vative drugs,  e.g.,  those  categorized  by  FDA  as 
1A.  The  use  of  these  mechanisms  also  eliminates 
the  need  for  sponsors  to  conduct  duplicative 


clinical  trials.  Thus,  manufacturers'  expenses 
associated  with  entering  some  existing  drug 
product  markets  can  be  reduced. 

Removal  of  Diethylstilbestrol  (DES)  From 
Animal  Feed:  The  estrogenic  compound  DES 
was  used  by  poultry  and  other  livestock  raisers 
for  many  years  to  increase  the  muscle  and  fat 
content  of  their  animals.  FDA  first  approved 
such  use  of  DES  in  1954. 6  After  that,  DES  was 
used  in  animal  feed  to  reduce  the  total  feed 
intake  necessary  to  achieve  maximum  animal 
weight  gain.  Furthermore,  some  studies  showed 
that  the  use  of  DES  in  animal  feed  helped  reduce 
the  cost  of  raising  poultry  and  other  livestock. 
Theoretically,  therefore,  the  use  of  DES  may 
have  helped  contain  poultry  and  meat  prices  for 
consumers. 

Ingestion  of  DES  by  humans,  however,  was 
eventually  correlated  with  an  above-normal  risk 
of  developing  certain  types  of  cancer,  such  as 
adenocarcinoma  of  the  vagina  or  the  cervix.7 
Because  of  DES's  cancer-producing  potential, 
FDA  attempted  several  times  to  remove  this 
substance  from  animal  feed. 

The  authority  for  its  efforts  was  the  Delaney 
clause,8  a  proviso  added  to  the  1958  food  addi- 
tive amendment  to  the  Food,  Drug,  and  Cos- 
metic Act: 

...  no  additive  shall  be  deemed  to  be  safe  if 
found  to  induce  cancer  when  ingested  by  man  or 
animal  .  .  . 

This  clause  was  reiterated  in  the  1968  animal 
drug  amendments  to  the  same  Act.9  Under  the 
animal  drug  amendments,  approvals  for  the  use 
of  new  animal  drugs,  which  are  used  in  the  live- 
stock industry  for  the  treatment  and  prevention 
of  disease  and  as  growth  promoters,  are  to  be 
granted  only  after  a  two-part  evaluation  by 
FDA.  First,  FDA  must  determine  that  the  drug  is 
safe  and  effective  for  use  in  animals.  Second,  it 
must  assess  the  safety  of  potential  residues  that 
might  occur  in  food  derived  from  treated  ani- 
mals.10 

"44  FR  54853. 
744  FR  54852. 

"Sec.  409(c)(3)(A),  Food,  Drug,  and  Cosmetic  Act,  21  U.S.C. 
348(c)(3)(A). 

'Sec.  512,  21  U.S.C.  360b. 
10Sec.  512,  23  U.S.C.  360b(d)(l). 
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After  evaluating  and  finding  DES  to  be  a  car- 
cinogenic substance,  in  1972,  FDA  attempted  to 
ban  its  use  in  animal  feed.  That  effort  termi- 
nated in  a  litany  of  legal  proceedings  with  the 
livestock  industry  and  animal  feed  manufactur- 
ers.11 On  January  12,  1976,  therefore,  FDA  once 
again  initiated  a  legal  process  to  remove  DES 
from  animal  feed.  This  time,  the  Agency  also 
issued  an  inflation  impact  statement  regarding 
its  proposed  ban  of  the  use  of  DES.12  The  state- 
ment concluded: 

1.  There  are  no  satisfactory  alternatives  to 
the  Agency's  proposed  action  which  are 
consistent  with  the  legal  constraints  im- 
posed by  the  Federal  Food,  Drug,  and 
Cosmetic  Act  as  amended. 

2.  Operating  expenses  to  feedlot  producers 
of  cattle  for  feed  and  other  items  are  esti- 
mated to  decrease  by  $156  million  during 
the  first  year  following  a  DES  ban.  These 
increased  costs  are  expected  to  fall  sub- 
stantially over  the  longer  term  as  substi- 
tutes to  DES  become  available  in  greater 
quantities. 

3.  Retail  prices  of  beef  are  estimated  to  rise 
by  about  2  cents  per  pound,  meaning  the 
per  capita  cost  of  beef  to  consumers  at 
current  levels  of  consumption  would  in- 
crease from  $2  to  $3  annually.  The  aggre- 
gate consumer  impact  is  estimated  at  $503 
million. 

4.  A  ban  on  DES  would  not  cause  major  in- 
flation impacts,  as  defined  by  the  HEW/ 
OMB  criteria,  in  the  areas  of  competition, 
productivity,  supplies  of  materials,  or  use 
of  energy. 

5.  The  benefits  from  implementing  the  pro- 
posed action  will  be  the  elimination  of  any 
risk  of  any  cancer  associated  with  the  con- 
sumption, via  the  edible  tissues  of  food- 
producing  animals,  of  residues  attributa- 
ble to  DES. 

According  to  former  FDA  Chief  Counsel, 
Richard  Cooper,  FDA's  mandate  under  the  Del- 
aney  clause  and  the  1968  animal  drug  amend- 
ments prevented  the  Agency  from  considering 
any  economic  benefits  of  DES  in  its  decision  to 
ban  the  drug  from  use  in  animal  feed  (127a). 


u44  FR  54854. 

,2FDA  docket  No.  76N-002,  Inflation  Impact  Statement  DES, 
Office  of  Planning  and  Evaluation,  January  1976. 


On  June  29,  1979,  then  FDA  Commissioner 
Donald  Kennedy  ordered  the  withdrawal  of  ap- 
proval of  the  new  animal  drug  application  for 
DES.13  In  taking  this  action  to  ban  the  use  of 
DES,  Commissioner  Kennedy  stated: 

FDA  is  not  authorized,  under  the  Food,  Drug 
and  Cosmetic  Act,  in  considering  the  question  of 
whether  a  new  animal  drug  has  been  shown  to 
be  safe  for  use,  to  weigh  the  "socio-economic" 
benefits  that  that  drug  provides  against  a  health 
risk  to  the  ultimate  human  consumers  of  treated 
animals.  Even  were  I  to  attempt  to  weigh  the 
benefits  of  DES  against  its  risks,  this  record 
would  not  provide  sufficient  information  to 
compute  the  risk  associated  with  DES  or  to  de- 
termine whether,  and  to  what  extent,  use  of  DES 
provides  any  health  benefit  or  even  any  econom- 
ic benefit  to  society. 

This  case  study  illustrates  one  situation  in 
which  FDA  interpreted  the  Delaney  clause  of  the 
Food,  Drug,  and  Cosmetic  Act  (its  primary  stat- 
utory authority)  to  mean  that  in  its  decisions 
concerning  the  removal  of  carcinogenic  sub- 
stances from  the  market,  it  cannot  consider  the 
results  of  economic  analyses. 

Evaluation  of  an  X-Ray  Equipment  Perform- 
ance Standard  Established  by  the  Bureau  of  Ra- 
diological Health  (BRH):  This  case  illustrates 
FDA's  use  of  cost-benefit  analysis  to  evaluate 
one  of  its  regulations. 

In  1972,  FDA's  BRH  promulgated  regulations 
containing  a  performance  standard  for  medical 
diagnostic  X-ray  systems  and  their  major  com- 
ponents.14 These  regulations,  among  other 
things,  required  X-ray  equipment  manufactur- 
ers to  certify  that  all  specified  components  man- 
ufactured after  August  1,  1979,  comply  with  the 
new  standard.  Another  provision  stated  that 
after  August  1,  1974,  no  uncertified  components 
could  be  assembled  or  reassembled  into  an  X- 
ray  system.15  The  purpose  of  these  regulations 
was  to  help  ensure  the  radiation  safety  of  diag- 
nostic X-ray  systems.16 

In  1974,  BRH  estimated  the  costs  and  benefits 
associated  with  these  regulations.  The  Bureau 

l344  FR  54900. 

"21  CFR  1020.30,  1020.31,  1020.32. 
"21  CFR  100.16. 
"44  FR  22755. 
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estimated,  for  example,  the  expected  numbers 
and  costs  of  X-ray  systems  which  would  be  mar- 
keted under  the  new  performance  standards. 
Upon  reviewing  3  years  of  experience  with  the 
regulations  for  diagnostic  X-ray  systems,  BRH 
realized  that  their  earlier  estimates  of  certain 
costs  and  benefits  were  incorrect.  In  1978,  using 
data  not  previously  available,  BRH  conducted  a 
second  analysis  of  costs  and  benefits  derived 
from  the  1972  regulations.  On  the  basis  of  this 
second  analysis,  BRH  concluded:17 

1.  The  provisions  of  section  1000.16  that  are 
effective  after  August  1,  1979,  will  affect 
only  a  small  fraction  of  the  uncertified  X- 
ray  systems  currently  in  use. 

2.  The  total  impact  of  this  regulation,  in 
terms  of  increased  cost  for  the  X-ray 
equipment  or  interruption  of  health  care 
delivery,  may  be  significant. 

3.  For  those  uncertified  systems  that  would 
be  affected,  by  virtue  of  their  sale  and  re- 
location, section  1000.16  is  not  likely  to 
be  a  cost-effective  approach  to  improve 
the  radiation  safety  performance  of  X-ray 
systems. 

Based  on  these  conclusions,  BRH,  through  the 
rulemaking  process,  modified  its  earlier  regula- 
tions by,  among  other  things,  permitting  the  in- 
stallation of  uncertified  components  into  exist- 
ing systems  whose  components  are  all  uncerti- 
fied and  permitting  the  continued  reassembly  of 
uncertified  equipment.  It  also  clarified  certain 
aspects  of  its  performance  standard  for  X-ray 
systems.18 

17lbid. 

1844  FR  49667. 


Incorporating  Costs-Savings  Studies  Into 
Vaccine  Approval  Decisions:  In  its  recently 
published  report  on  viral  and  rickettsial  vac- 
cines, FDA's  Panel  of  Review  of  Viral  and  Rick- 
ettsial Vaccines  identified  economics  as  a  major 
consideration  in  the  evaluation  of  vaccines.19 
The  panel  noted  that  because  adequate  data  are 
often  lacking,  economic  considerations  are 
often  ignored  in  such  evaluations. 

The  panel  stated: 

These  data  would  contrast  the  cost  of  a  vac- 
cine and  its  administration  plus  the  costs  (medi- 
cal care,  rehabilitation,  impairment  of  ability  to 
earn  income)  of  vaccine-related  disease  with 
costs  of  a  similar  nature  that  would  have  ac- 
crued from  cases  of  the  natural  disease. 

The  panel  further  stated: 

For  an  "acceptably  safe"  and  effective  vaccine 
against  a  serious  disease,  .  .  .  the  ratio  should 
be  highly  favorable.  However,  if  the  preventable 
disease  occurs  chiefly  in  young  children  and  is 
infrequently  associated  with  permanent  seque- 
lae, a  different  answer  might  result.  The  ques- 
tion then  might  become  "how  much  cost  can  be 
justified  to  prevent  one  crippling  or  lethal  case  of 
disease?"  This  clearly  requires  societal  rather 
than  scientific  judgment. 

This  panel  has  strictly  an  advisory  capacity 
within  FDA,  and  its  reports  do  not  reflect  offi- 
cial FDA  policy.  The  panel's  discussion  of  eco- 
nomics in  the  vaccine  evaluation  process,  how- 
ever, does  illustrate  some  concern  among  FDA's 
advisors  for  evaluating  the  potential  economic 
impacts  of  vaccines. 

1945  FR  25665. 
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In  order  to  facilitate  the  discussion  presented 
in  the  next  section  of  this  chapter  on  the  implica- 
tions of  including  cost-effectiveness  criteria  in 
the  market  approval  process,  OTA  has  devel- 
oped the  hypothetical  model  of  a  cost-effective- 
ness analysis  (CEA)  outlined  below.  It  is  very 
important  to  keep  in  mind  that  the  use  of  the 
model  for  illustrative  purposes  does  not  negate 
or  diminish  any  of  the  weaknesses  or  possibil- 
ities for  misuse  involved  in  formal  cost-effec- 


tiveness analyses/cost-benefit  analyses  (CEA/ 
CBAs)  that  seek  a  single,  quantitative  bottom- 
line,  such  as  a  cost-effectiveness  ratio.  Such  a 
ratio  is  used  in  the  model  because  1)  it  points  out 
some  of  the  data  and  other  problems  related  to 
use  of  bottom-lines,  and  2)  it  is  possible,  given 
the  quantitative  nature  of  FDA's  regulatory 
process,  that  use  of  a  CEA  ratio  might  be  seri- 
ously considered  by  the  agency  if  CEA  were 
added  to  its  mandate. 
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OTA's  model  is  simplistic  by  intent  and  does 
not  address  the  numerous  assumptions  and  vari- 
ations that  are  possible.  Several  caveats  and 
assumptions  should  accompany  an  analysis 
such  as  the  one  presented.  Potential  method- 
ological problems  associated  with  the  use  of 
CEA  in  general  are  explained  in  detail  in  a 
background  paper  of  this  assessment.20  Addi- 
tional problems  with  its  use  in  market  approval 
processes  are  explained  below.  One  problem, 
for  example,  is  that  data  for  some  important 
variables  would  be  difficult,  if  not  impossible, 
to  obtain. 

In  assessing  a  drug's  cost  effectiveness,  FDA 
might  first  assess  the  agent's  efficacy  and  safety 
and  quantify  its  effects  into  measurable  units  of 
"net  health  effect."  Let  us  assume,  for  example, 
that  an  NDA  is  submitted  for  applicant  drug 
"D,"  which  is  used  to  treat  high  blood  pressure 
(hypertension).  Let  us  also  assume  that  in  pre- 
marketing clinical  trials,  drug  "D"  consistently 
lowered  by  10  percent  blood  pressure  in  50  per- 
cent of  tested  hypertensive  patients,  and  that 
such  a  drop  in  pressure  could  be  correlated  with 
a  5-percent  reduction  in  morbidity  and  mortali- 
ty (e.g.,  heart  attacks,  strokes,  and  kidney  dis- 
ease). Suppose  drug  "D"  also  produced  undesir- 
able side  effects  that  in  premarketing  clinical 
trials  accounted  for  a  2-percent  increase  in  days 
of  disability  in  the  tested  population.  Let  us  also 
assume  that  the  tested  population  accurately 
represents  the  general  hypertensive  population. 

To  calculate  the  "net  health  effect"  of  drug 
"D,"  all  such  positive  effects  (e.g.,  5-percent 
reduction  in  mortality)  and  negative  effects 
(e.g.,  2-percent  increase  in  disability  caused  by 
side  effects)  would  have  to  be  converted  into  a 
uniform  and  measurable  unit  of  health.21  The 
"net  health  effect"  of  drug  "D"  could  then  be 
calculated  for  each  indication  listed  on  the  pro- 
posed drug  labeling  (e.g.,  treatment  of  moderate 
hypertension  in  ambulant  patients).  Specific 
conditions  of  use  (e.g.,  in  conjunction  with 
other  drugs)  and  peculiar  effects  in  special  pop- 


2< 'Background  Paper  #1:  Methodological  Issues  and  Literature 
Review,  prepared  by  OTA. 

21As  discussed  in  Background  Paper  #2:  Methodological  Issues 
and  Literature  Review,  some  analysts  have  used  the  unit  termed 
quality-adjusted  life  year  for  this  purpose  (652). 


ulations  (e.g.,  the  elderly)  could  be  accounted 
for  in  the  calculation  of  net  health  effects. 

Once  the  drug's  effects  were  converted  into 
measurable  units  of  "net  health  effects,"  the  "net 
cost"  of  achieving  a  desirable  level  of  health  ef- 
fect (e.g.,  a  5-percent  reduction  in  mortality  and 
morbidity)  would  have  to  be  calculated.  A  "net 
cost"  could  include  such  items  as  the  cost  of  pur- 
chasing the  drug,  the  cost  of  treating  drug-in- 
duced side  effects,  and  perhaps  the  cost  of  treat- 
ing other  illnesses  in  the  persons  whose  lives  are 
saved  by  the  use  of  drug  "D."  Subtracted  from 
such  costs  could  be  savings  resulting  from  any 
reduced  costs  of  hypertension  treatment  (e.g., 
lower  use  of  previously  approved  drugs  or  de- 
cline in  hospitalizations  and  in  physician  office 
visits)  resulting  from  the  use  of  applicant  drug 
"D." 

At  the  conclusion  of  this  phase  of  the  anal- 
ysis, one  could  construct  a  ratio  of  net  cost  in 
dollars  to  one  unit  of  "net  health  effect" 
achieved  through  the  use  of  drug  "D"  in  the 
treatment  of  hypertension,  e.g.: 


Cost-effectiveness  ratio  = 


Net  cost 


Unit  of  net  health  effect 


In  the  next  phase  of  the  analysis,  the  net  cost 
(in  dollars)  of  achieving  a  desired  net  health  ef- 
fect through  the  use  of  drug  "D"  would  be  com- 
pared with  the  net  cost  of  achieving  the  same  net 
health  effect  by  using  an  existing  approved 
treatment  modality  (e.g.,  another  drug,  sur- 
gery, or  biofeedback)  to  lower  blood  pressure. 
Such  a  comparison  of  costs  would  require  that 
cost-effectiveness  ratios,  i.e.,  net  cost  (in  dol- 
lars) /unit  of  net  health  effect,  be  derived  for 
each  hypertension  treatment  modality.  The  fol- 
lowing hypothetical  ratios  for  example,  might 
be  derived: 


Hypertension  treatment 

modality 
Applicant  drug  "D".  :  .  . 
Approved  drug  "A"  .  .  . 
Surgical  procedure  "X"  . 
Biofeedback  


Net  cost/uniform  unit 
of  net  health  effect 
$400 
$250 
$3,000 
$100 


Let  us  assume  that  no  other  treatment  of  hyper- 
tension was  available  at  the  time  of  the  analysis. 

The  final  phase  of  such  an  analysis  would  be 
to  establish  criteria  for  judging  the  cost-effec- 
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tiveness  ratios  for  each  treatment  modality  and 
for  using  such  ratios  to  help  determine  if  appli- 
cant drug  "D"  should  be  approved.  In  the  exam- 
ple above,  if  a  criterion  for  market  approval 
were  that  the  applicant  drug  had  to  produce  one 
unit  of  "net  health  effect"  at  a  cost  lower  than 
the  cost  of  using  any  other  approved  antihyper- 
tensive drug,  then  drug  "D"  might  not  be  ap- 
proved. If  such  a  criterion  included  comparison 
with  other  approved  forms  of  hypertension 
treatment,  then  drug  "D"  would  presumably  be 
approved  if  compared  with  surgery,  although  it 
might  not  be  approved  if  compared  with  bio- 
feedback. When  one  considers  in  this  hypotheti- 
cal evaluation  process  the  need  for  a  variety  of  • 
available  treatment  modalities  to  meet  individ- 
ualized patient  situations,  then  the  relatively 
small  differences  in  cost  effectiveness  between 
applicant  drug  D,  biofeedback,  and  approved 
drug  A  become  much  less  important. 

Cost  effectiveness  might  be  used  in  formal  re- 
view processes  of  drugs  that  have  been  mar- 
keted for  a  number  of  years,  instead  of  those  for 
new  drugs.  FDA  conducts  at  least  three  such  re- 


view processes,  and  to  date,  the  Agency  has  not 
used  cost  effectiveness  in  any  of  them.  First,  in 
its  drug  efficacy  study  implementation  project, 
FDA  is  conducting  a  one-time  review  of  the  ef- 
ficacy of  drug  products  approved  between  1938 
and  1962  (131).  The  Agency  categorizes  prod- 
ucts according  to  their  documented  clinical  ef- 
ficacy, has  removed  from  the  market  some 
products  for  which  efficacy  documentation  is 
lacking  (e.g.,  selected  fixed-dosage  antibiotic 
combination  products),  and  is  attempting  to  re- 
move other  such  products.  Second,  in  its  Over- 
the-Counter  (OTC)  Drug  Review  Program, 
FDA  is  conducting  a  one-time  review  of  the  safe- 
ty and  efficacy  of  several  hundred  ingredients 
used  in  OTC  or  nonprescription  drug  products. 
Third,  FDA  (technically,  the  Secretary  of  Health 
and  Human  Services)  has  authority  to  remove 
from  the  market  a  drug  that  represents  an  "im- 
minent hazard"  to  the  public's  health.  When  this 
authority  was  exercised  to  remove  the  antidia- 
betic drug  phenformin  from  the  market,  FDA 
extensively  reviewed  the  drug's  safety  and  to 
some  extent  its  efficacy. 


IMPLICATIONS  OF  CEA/CBA  IN  MARKET  APPROVAL: 
GENERAL  FINDINGS 


As  described  in  chapter  2,  CEA  is  an  analyti- 
cal device  that  can  be  used  to  help  decision- 
makers allocate  resources,  usually  in  the  public 
sector.  The  primary  purpose  of  FDA's  drug  and 
medical  device  market  approval  processes  is  to 
protect  the  public  from  unsafe  and  ineffective 
products.  FDA's  marketing  approval  processes 
do,  however,  indirectly  influence  the  allocation 
of  public  resources  in  at  least  two  ways.  First, 
the  types  of  regulations  and  procedures  estab- 
lished by  FDA  affect  the  allocation  of  its  operat- 
ing budget.  Second,  many — if  not  most — of  the 
drugs  and  medical  devices  approved  by  FDA 
eventually  are  used  in  health  care  services  that 
are  paid  for  through  publicly  financed  pro- 
grams, such  as  medicare  and  medicaid. 

The  use  of  cost  effectiveness  as  a  criterion  in 
FDA's  market  approval  processes  for  drugs  and 
medical  devices  would  require  a  substantial  shift 


in  the  Federal  Government's  approach  to  regu- 
lating the  medical  care  marketplace.  At  present, 
no  Government  effort  is  designed  explicitly  to 
reduce  medical  expenditures  by  directly  slowing 
down  or  stopping  the  market  introduction  of 
medical  goods  and  services.  The  Federal  Gov- 
ernment does  not  extensively  evaluate  the  cost 
effectiveness  of  drugs  or  medical  devices  in  any 
of  its  efforts  to  contain  health  care  costs.  Con- 
ceivably, such  evaluation  could  take  place  in  the 
National  Center  for  Health  Care  Technology, 
which  advises  the  Health  Care  Financing  Ad- 
ministration (HCFA)  on  reimbursement  policies 
for  selected  medical  technologies.  Results  of 
CEAs  involving  drugs  and  medical  devices 
could  be  incorporated  into  HCFA's  policies  for 
reimbursing  the  use  of  selected  drugs  and  medi- 
cal technologies  under  medicare,  medicaid,  and 
any  other  federally  operated  health  insurance 
program. 


Ch.  8— Market  Approval  for  Drugs  and  Medical  Devices  •  95 


Potential  Positive  Effects 

There  are  at  least  two  hypothetical  positive 
effects  of  incorporating  CEA  into  FDA's  market 
approval  process  for  drugs  and  devices. 

First,  manufacturers  could  formally  incorpo- 
rate cost-effectiveness  criteria,  based  on  societal 
values,  into  their  research,  development,  and 
marketing  strategies.  Manufacturers  most  likely 
do  use  some  form  of  CEA/CBA  to  allocate  their 
R&D  expenditures.  The  primary  criteria  used  in 
such  allocations,  however,  may  emphasize  such 
items  as  total  sales,  market  portions,  and  re- 
turn-on-investment.  If  FDA  used  criteria  such  as 
reduced  treatment  costs,  improved  levels  of 
health,  and  improved  efficiency  in  disease  pre- 
vention or  treatment,  then  more  manufacturers 
might  develop  new  products  and  seek  new  mar- 
kets where  existing  treatment  or  prevention 
measures  are  ineffective  or  inefficient.  Some  in- 
dustry representatives  claim  that  the  leading 
drug  research  firms  currently  include  such  pub- 
lic health  criteria  in  their  research  priorities.  The 
existence  of  orphan  drugs  (i.e.,  existing  agents 
for  which  there  is  a  small,  demonstrated  clinical 
need  but  no  manufacturer),  however,  indicates 
that  at  least  for  certain  products,  profitability 
takes  precedence  over  societal  need.  Conceiv- 
ably, such  use  of  CEA  could  help  the  Govern- 
ment encourage  participation  by  drug  and  medi- 
cal device  manufacturers  in  public  efforts  to  de- 
velop more  cost-effective  medical  technologies. 

Second,  if  FDA  were  able  to  accurately  assess 
the  cost  effectiveness  of  an  applicant  medical 
device  or  drug,  the  Agency  might  help  reduce 
expenditures  for  inefficient  products  by  keeping 
them  off  the  market  entirely.  By  evaluating  the 
cost  effectiveness  of  medical  devices  and  drugs 
in  the  market  approval  process,  the  Govern- 
ment would  be  assessing  a  product  very  early  in 
its  diffusion  process.  Perhaps  this  early  evalua- 
tion process  would  lead  to  better  direction — and 
perhaps  an  expansion — of  experimentation  with 
new  drugs  and  devices.  At  present,  substantial 
non-FDA-approved  clinical  experimentation 
with  newly  approved  drugs  takes  place  in  un- 
controlled situations.  Such  early  evaluation, 
however,  would  have  no  effect  on  the  inefficient 
use  of  drugs  and  medical  devices  found  to  be 
cost  effective  in  clinical  trials. 


Potential  Problems 

Among  the  consumer  advocates,  FDA  em- 
ployees, and  representatives  of  the  pharmaceu- 
tical industry  surveyed  in  this  assessment,  there 
appeared  to  be  widespread  agreement  that  cost 
effectiveness  is  not  an  appropriate  criterion  to 
use  in  the  drug  and  medical  device  approval 
process.  Using  information  and  ideas  obtained 
from  several  individuals,  OTA  developed  the 
following  analysis  of  potential  problems. 

First,  the  market  approval  process  may  be  too 
early  in  the  life  of  a  drug  or  medical  device  to 
evaluate  its  cost  effectiveness;  information  re- 
garding a  product's  safety  and  efficacy  are  usu- 
ally available  for  only  one  or  two  indications  of 
use  at  the  time  a  sponsor  submits  an  initial  NDA 
for  a  new  drug  or  an  application  for  approval  of 
a  new  device.  Data  regarding  a  drug's  safety  and 
efficacy  in  medical  conditions  not  listed  in  the 
product's  official  labeling  are  not  often  gener- 
ated until  a  product  has  been  marketed  for  at 
least  a  few  years.  If  a  new  drug  or  medical  de- 
vice were  not  approved  for  marketing  because 
its  cost  effectiveness  did  not  compare  favorably 
with  already  approved  products,  then  the  new 
product  might  never  be  fully  evaluated,  particu- 
larly in  the  treatment  of  medical  problems  other 
than  the  one(s)  studied  initially.  The  total  bene- 
fits, risks,  and  cost  effectiveness  of  drugs  that 
are  used  in  the  treatment  of  more  than  one  medi- 
cal problem  often  may  not  be  known  for  several 
years  after  the  product's  initial  development. 
Examples  of  such  products  include  propranolol 
used  in  the  treatment  of  angina,  migraine  head- 
aches, and  hypertension;  selected  antibiotics 
used  in  the  treatment  of  acne;  amantadine  used 
in  the  treatment  of  parkinsonism;  and  phenyto- 
in  used  in  the  treatment  of  certain  cardiac  ar- 
rhythmias. In  addition,  rare  or  delayed-onset 
adverse  drug  reactions,  for  example,  drug-in- 
duced cancers,  would  not  likely  be  included  in 
premarketing  CEA. 

James  B.  Russo,  SmithKline  Corp.,  offers  an 
industrial  concern  about  the  use  of  CBA  in  the 
market  approval  process  for  drugs  (529): 

Our  primary  concern  is  over  the  fact  that  cost- 
benefit  analysis  in  the  drug  area  is  a  new  disci- 
pline, and  one  which  simply  must  not  be  widely 
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applied,  particularly  in  the  area  of  new  drug  ap- 
proval, until  we  have  real  confidence  in  its  pre- 
dictive reliability  ...  I  don't  really  know  how 
to  state  that  point  as  strongly  as  I'd  like  to. 
Think  back  to  probenecid.  The  drug  was  devel- 
oped to  slow  the  excretion  of  penicillin  from  the 
kidney,  because  in  those  days  penicillin  produc- 
tion was  inadequate.  By  the  time  probenecid 
was  shown  to  be  safe  and  effective  for  that  pur- 
pose, penicillin  was  coming  out  of  the  industry's 
ears.  Had  that  NDA  been  looked  at  on  the  basis 
of  the  product's  cost  against  its  possible  benefits 
in  prolonging  blood  levels  of  penicillin,  work  on 
the  drug  would  have  been  stopped  in  the  early 
'50s.  Of  course,  once  it  was  found  that  it  speeded 
the  excretion  of  uric  acid,  an  entirely  new  and  a 
relatively  important  means  of  relieving  gout  be- 
came a  possibility.  It  isn't  simply  a  matter  of 
NDA  approval.  If  we  knew  we  would  have  to 
pass  that  kind  of  test  at  the  NDA  stage,  I  fear 
that  a  lot  of  projects  would  be  cancelled  long  be- 
fore we  had  practical  information  on  the  drug's 
full  potential. 

In  recent  years,  however,  FDA  has  approved 
very  few  NDAs  to  add  to  a  previously  approved 
product's  official  labeling  a  new  major  indica- 
tion that  required  data  from  new  clinical  trials. 
Between  January  1,  1974,  and  September  30, 
1979,  for  example,  FDA  approved  4  such  NDAs 
out  of  a  total  of  484  total  NDA  approvals  for  all 
reasons  (221).  There  are,  however,  362  active 
commercial  INDs  for  products  being  used  for  in- 
dications not  currently  listed  in  the  products'  of- 
ficial labeling.  In  addition,  for  perhaps  hun- 
dreds of  approved  products,  FDA  has  permitted 
manufacturers,  through  the  supplemental  NDA 
process,  to  make  minor  modifications  (i.e,  those 
not  requiring  manufacturers  to  sponsor  new 
clinical  trials)  in  their  products'  official  indica- 
tions for  use.  The  extent  to  which  marketed 
drugs  are  used  for  unapproved  indications  is  not 
known.  Manufacturers  often  have  no  economic 
incentives  to  seek  FDA  approval  for  popular, 
unofficial  clinical  uses  of  their  products. 

Second,  the  calculation  of  costs  needed  to 
assess  the  cost  effectiveness  of  a  new  drug  would 
require  FDA  to  either:  1)  use  existing  estimates 
of  costs  of  treating  illnesses  using  alternative 
forms  of  therapy,  or  2)  generate  new  data  bases. 
Problems  encountered  in  using  existing  data 
bases  are  discussed  in  a  background  paper  of 


this  assessment.22  At  present,  FDA  has  no  intra- 
mural capability  for  generating  new  data  re- 
garding the  treatment  costs  for  selected  diseases. 
Conceivably,  FDA  could  ask  or  require  drug 
and  medical  device  manufacturers'  to  submit 
estimates  of  the  costs  of  treatments  provided 
during  the  clinical  testing  of  a  new  drug.  Mean- 
ingful estimates  of  treatment  costs  might  be  dif- 
ficult to  calculate,  however,  because  the  prices 
charged  for  medical  care  are  dynamic  and  can 
vary  substantially  among  geographical  regions. 

At  the  time  FDA  approves  a  drug  or  medical 
device,  it  does  not  know — nor  can  it  influence — 
how  much  a  manufacturer  will  charge  for  its 
newly  approved  product.  FDA  could  ask  a  man- 
ufacturer to  estimate  a  product's  selling  price. 
Apparently,  however,  this  task  would  be  diffi- 
cult for  manufacturers  to  accomplish  at  the  time 
of  market  approval.  Further,  the  price  of  a  drug 
or  medical  device  will  likely  change  over  time, 
and  cost-effectiveness  information  calculated  at 
the  time  of  market  approval  would  likely 
change. 

Third,  the  effect  on  competition  of  using  cost 
effectiveness  as  a  criterion  for  market  approval 
of  drugs  and  medical  devices  is  unknown.  Po- 
tential implications  vary  substantially  depend- 
ing on  how  FDA  might  use  the  cost-effectiveness 
criterion. 

If  FDA  assumed  the  responsibility  for  con- 
ducting CEAs,  theoretically  manufacturers 
would  not  be  burdened  with  the  expense  of  de- 
veloping the  capability  to  conduct  such  anal- 
yses. Most  research-based  companies,  however, 
would  likely  establish  their  own  capabilities;  at 
least  two  pharmaceutical  firms  currently  per- 
form CEAs  on  some  of  their  products.  One  com- 
pany, Merck  Sharp  and  Dohme,  developed  a 
computerized  cost-benefit  model  to  illustrate  the 
costs  and  benefits  of  its  pneumococcal  vaccine 
(464).  Another  pharmaceutical  manufacturer, 
SmithKline  and  French,  has  extensively  studied 
the  costs  and  benefits  of  the  use  of  one  of  its 
drugs,  cimetidine,  in  different  populations 
(529).  If  FDA  required  each  sponsor  of  every 


"Background  Paper  #1:  Methodological  Issues  and  Literature 
Review,  prepared  by  OTA. 
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new  drug  or  medical  device  to  conduct  CEA  as  a 
part  of  the  premarketing  approval  processes  for 
drugs  and  devices,  then  the  use  of  cost  effec- 
tiveness would  add  to  the  financial  and  regula- 
tory hurdles  of  new  product  introduction.  Con- 
ceivably, that  situation  could  reduce  manufac- 
turers' willingness  to  bring  a  new  product  to 
market  and  could  reduce  competition. 

If  FDA  required  only  the  original  manufac- 
turer of  a  new  drug  or  medical  device  (i.e.,  one 
not  previously  marketed  in  the  United  States)  to 
analyze  the  cost  effectiveness  of  its  product, 
then  the  burden  of  conducting  such  analyses 
would  fall  primarily  on  the  leading  innovating 
companies. 

If  FDA  used  cost  effectiveness  as  a  criterion 
for  the  marketing  of  only  new  therapeutic  enti- 
ties, and  not  for  either  generic  or  "me  too"  prod- 
ucts,23 and  if  manufacturers  perceived  such  a 
task  as  too  expensive,  then  manufacturers  might 
attempt  to  break  into  existing  markets  for  multi- 
ple source  products  rather  than  to  develop  new 
drugs  and  create  new  markets. 

If  FDA  applied  cost-effectiveness  criteria  to 
"me  too"  type  products,  then  a  manufacturer 
might  lower  the  introductory  price  of  its  new 
product  in  order  to  make  it  compare  favorably 
to  already-approved  products  in  the  same  thera- 
peutic category.  Once  a  product  was  marketed, 
however,  its  manufacturer  could  alter  its  price, 
and  FDA  has  no  authority  to  control  the  price  of 
approved  drugs  and  medical  devices.  Excessive 
increases  in  postapproval  prices  would  likely  be 
limited  by  competition  in  the  marketplace  for 
similar  products. 

Fourth,  the  use  of  CEA  to  evaluate  new  drugs 
and  medical  devices  would  require  extensive  re- 
sources, substantial  time,  and  creative  applica- 
tion of  existing  data.  FDA  would  likely  have  to 
compare  the  relative  safety,  efficacy,  and  cost  of 
an  applicant  product  with  those  same  character- 
istics of  marketed  products.  In  order  to  perform 
such  comparisons,  FDA  would  have  to  do  the 
following: 


""Me  too"  products  are  those  that  are  chemically  dissimilar 
enough  to  approved  products  to  warrant  a  separate  NDA  review, 
but  that  nevertheless  produce  quite  similar  net  therapeutic  effects. 


•  quantitatively  assess  the  safety  (risks)  and 
efficacy  (benefits)  of  each  marketed  drug 
and  medical  device; 

•  establish  standards  for  clinical  efficacy, 
safety,  and  cost  of  each  available  form  of 
treatment — and  possibly  prevention — in 
numerous  disease  states  (note:  such  stand- 
ards could  be  incorporated  into  a  mono- 
graph system  such  as  those  used  for  OTC 
drug  products  and  antibiotics);  and 

•  calculate  cost-effectiveness  information  for 
the  use  of  each  drug  and  medical  device  in 
specialized  populations,  such  as  the  elderly 
and  persons  with  specific  medical  prob- 
lems. 

Although  it  is  debatable  whether  FDA  has 
statutory  authority  to  evaluate  new  products 
relative  to  currently  marketed  products,  a  pro- 
vision in  the  Senate-passed  version  of  the  Drug 
Regulation  Reform  Act  of  1979  would  allow 
FDA  to  consider  formally  and  explicitly  the 
"benefits  and  risks  of  available  therapies"  when 
evaluating  applicant  drugs  in  the  NDA  review 
process. 

Fifth,  the  cost  effectiveness  of  different  prod- 
ucts would  vary  substantially  depending  on  the 
forms  of  treatment  being  compared.  A  compari- 
son between  a  new  drug  and  a  surgical  proce- 
dure in  the  treatment  of  a  selected  medical  prob- 
lem, for  example,  could  yield  large  differences 
in  cost-effectiveness  ratios.  Such  a  comparison 
might  be  useful  to  FDA  in  its  evaluation  of  the 
drug.  The  information  yielded  could  be  helpful 
in  the  evaluation  of  the  new  product,  especially 
if  the  product  were  a  new  chemical  entity  that 
represented  a  therapeutic  breakthrough.  Ulti- 
mate treatment  for  uncontrollable  malignant 
hypertension  in  a  young  person,  for  example, 
has  been  the  removal  of  one  or  both  kidneys 
(nephrectomy).  Until  recently,  many  cases  of 
this  disease  have  not  been  controllable  through 
the  use  of  drugs.  A  potent  new  antihypertensive 
drug,  minoxidil,  has  been  shown  to  effectively 
lower  very  high  blood  pressure,  and  in  some 
studies,  its  use  reduced  the  need  for  nephrec- 
tomies (482).  There  is  no  other  product  on  the 
market  that  resembles  minoxidil  either  chemi- 
cally or  therapeutically.  When  compared  to  sur- 
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gical  removal  of  the  kidneys,  minoxidil  would 
likely  be  quite  cost  effective  for  some  patients. 

Comparisons  between  two  drugs,  however, 
especially  two  drugs  with  similar  therapeutic  ef- 
fects, would  likely  yield  only  small  differences 
in  cost-effectiveness  ratios.  Small  differences 
would  be  of  little  value.  Thiazide  diuretics,  for 
example,  represent  another  form  of  antihyper- 
tensive drug  therapy.  There  are  approximately 
30  different  single-entity  thiazide  diuretic  prod- 
ucts on  the  market,  and  each  one  produces  very 
similar  therapeutic  and  adverse  effects.  In  cost- 
effectiveness  comparisons  among  these  30  prod- 
ucts, the  differences  in  cost-effectiveness  ratios 
would  likely  be  very  small;  hence  such  compari-  • 
sons  would  likely  be  of  little  distinguishing  val- 
ue in  FDA's  NDA  review  process  for  a  new  thia- 
zide diuretic. 

Sixth,  the  cost  effectiveness  of  a  drug  or  a 
medical  device  might  be  influenced  more  by  the 
conditions  of  its  use  than  by  its  demonstrated  ef- 
ficiency in  premarketing  clinical  trials.  Factors 
such  as  dosage  regimen,  route  of  administra- 
tion, and  palatability  (taste)  all  could  influence 
a  product's  acceptance  by  patients  and  hence  af- 
fect its  cost  effectiveness.  Even  if  FDA  were  able 
to  accurately  assess  the  cost  effectiveness  of  a 
drug  or  medical  device  at  the  time  of  market  ap- 
proval, such  an  assessment  might  not  accurately 
predict  the  efficiency  of  the  product  in  general 
use. 

Seventh,  if  FDA  used  cost  effectiveness  as  a 
criterion  in  its  drug  and  medical  device  market 


approval  processes,  the  Agency  would  have  to 
rely  on  speculative  and  uncertain  data  for  two 
important  valuations.  First,  FDA  measures  the 
safety  and  efficacy  of  drugs  and  medical  devices 
in  terms  of  changes  in  physiologic  functions,  not 
in  terms  of  changes  in  a  person's  overall  health 
status.  The  efficacy  of  an  antihypertensive  drug, 
for  example,  is  evaluated  on  the  degree  to  which 
the  drug  reduces  a  person's  blood  pressure.  FDA 
does  not  quantify  the  effect  a  drop  in  blood 
pressure  would  likely  produce  in  the  health 
status  of  a  hypertensive  patient.  Second,  FDA 
analysts  would  face  the  challenge  of  assessing 
the  economic  value  of  physiological  changes — 
and  if  possible  of  those  in  health  status — among 
persons  using  the  medical  device  or  drug. 

Eighth,  by  using  cost  effectiveness  as  a  cri- 
terion in  the  market  approval  process  for  drugs 
and  devices,  FDA  might  be  extending  its  role 
beyond  the  scope  of  responsibilities  Congress 
intended  the  Agency  to  have.  FDA's  primary 
purpose  is  to  protect  the  public  from  unsafe  and 
ineffective  drugs  and  medical  devices  and  from 
unsafe  cosmetics,  foods,  and  food  additives. 
Thus  far,  Congress  has  not  asked  FDA  to  use 
economic  criteria  to  regulate  the  choice  of  safe 
and  effective  products  during  the  market  ap- 
proval process.  Choices  based  on  economic  cri- 
teria are  left  to  be  made  by  consumers,  health 
care  practitioners,  hospital  administrators,  and 
private  and  public  health  insurance  carriers 
after  a  product  has  been  marketed. 
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9. 

R&D  Programs 


INTRODUCTION  AND  BACKGROUND 


The  Federal  Government  supports  a  wide 
range  of  health  R&D  activities.  The  object  of 
this  research,  no  matter  what  its  form,  is  the 
production  of  knowledge.  Some  research  is  de- 
signed to  yield  information  on  health  itself,  or 
on  diseases  and  disabling  conditions.  Some  is 
designed  to  produce  new  tools,  technologies,  to 
intervene  in  disease  processes,  or  to  counteract 
the  effects  of  disease.  Some  research  evaluates 
those  tools;  other  research  investigates  the  use 
of  technology  and  other  aspects  of  the  health 
care  delivery  systems.  Much  of  the  existing  re- 
search serves  multiple  purposes,  and  some 
yields  results  that  are  more  valuable  to  other 
fields  than  to  the  field  the  research  was  designed 
to  address. 

R&D  have  given  the  health  care  system  and 
this  country  a  great  deal  of  beneficial  informa- 
tion and  many  effective  technologies,  but  are  ac- 
tivities fraught  with  uncertainties.  These  ac- 
tivities can  also  be  expensive  ones — billions  of 
dollars  are  spent  on  health  R&D  each  year  in 
this  country  by  the  Federal  Government  alone. 
The  expenditure  of  Federal  funds  for  R&D  is  an 
investment  in  the  future — and  much  of  this  in- 
vestment represents  a  public  good.  Therefore,  it 
is  important  that  moneys  for  health  R&D  be 
spent  as  wisely  as  possible  and  in  accord  with  a 
balance  between  public  and  scientific  priorities. 

R&D  support  by  Federal  agencies  can  be 
viewed  from  two  perspectives:  process  and  sub- 
stantive. The  former  involves  questions  of  what 
methods  are  used  to  make  administrative  deci- 
sions regarding  selection  of  priorities,  program 
direction,  or  projects  to  be  funded  within  each 
agency.  The  issues  relate  to  selection  and  pur- 
suit of  goals  and  to  the  methods  employed  to  as- 
sist in  efforts  to  allocate  resources.  The  latter 
perspective,  "substantive"  performance,  in- 
volves questions  and  issues  pertaining  to  the 
agencies'  performance  or  sponsorship  of  re- 
search that  examines  the  allocation  of  techno- 


logical or  other  resources  in  the  health  care  sys- 
tem. 

Since  this  OTA  assessment  focuses  on  the  use 
of  cost-effectiveness  analysis/cost-benefit  anal- 
ysis (CEA/CBA),  the  chapter  concentrates  on 
questions  dealing  with  that  analytical  tech- 
nique. Four  Federal  agencies'  activities  in  regard 
to  CEA/CBA  from  both  of  the  aforementioned 
perspectives  are  examined.  If  an  agency  does 
not  support  or  use  CEA/CBA,  the  methods  and 
procedures  it  does  use  are  discussed  briefly.  Al- 
though the  distinction  between  the  process  and 
substantive  perspectives  of  research  support  is 
at  times  unclear,  an  attempt  is  made  in  the  dis- 
cussion below  to  separate  them  as  clearly  as 
possible  and  to  note  the  similarities  when  the 
situation  requires. 

Research  Classifications. — Health  R&D  is  an 
umbrella  term  used  to  describe  a  wide  range  and 
diversity  of  activities.  A  single  scientist  focusing 
on  an  extremely  narrow  topic  within  the  field  of 
biomedical  science  is  doing  health  research.  The 
same  can  be  said  of  the  analysts  that  are  examin- 
ing the  performance  of  a  patient  screening  pro- 
gram, developing  physician  productivity  meas- 
ures, or  working  on  any  number  of  other  health 
care  issues. 

To  help  describe  the  primary  focus  of  health- 
care-related R&D,  the  range  of  activities  is  often 
broken  into  four  loosely  defined  categories: 
1)  basic  research,  2)  applied  research,  3)  appli- 
cation (or  transfer)  research,  and  4)  health  serv- 
ices research.  The  demarcations  between  these 
four  research  categories  are  not  clearly  defined. 
Nevertheless,  the  classifications  play  an  impor- 
tant role  in  the  process  of  setting  health  care  re- 
search priorities,  allocating  and  distributing 
funds,  and  evaluating  outcomes  or  products  of 
R&D  efforts.  At  several  levels  in  the  health  care 
decisionmaking  and  policy  process,  the  in- 
tended purpose  of  a  given  research  effort  is  im- 
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portant.  There  is  constant  tension  in  the  deci- 
sionmaking process  between  those  who  advo- 
cate the  allocation  of  funds  for  increased  sup- 
port of  basic  research,  those  who  feel  more 
work  is  needed  in  applying  more  fully  the 
knowledge  and  technologies  that  exist  (applica- 
tion or  transfer  research),  and  those  who  cite  a 
pressing  need  to  examine  what  is  already  in 
place  and  how  well  it  is  working  or  how  to 
make  it  work  better  before  adding  more  to  the 
system  (services  research).  As  a  result  of  these 
different  perceived  research  needs,  the  research 
"label"  that  is  affixed  to  a  given  health  care  pro- 
gram or  initiative  can  be  quite  important  to  its 
ultimate  success  in  the  resource  allocation  proc- 
ess. 

For  purposes  of  the  discussion  below,  the  Fed- 
eral health  care  R&D  effort  is  divided  into  two 
broad  general  categories:  1)  biomedical  re- 
search, and  2)  health  services  research.  Biomedi- 
cal research  includes  basic,  applied,  and,  to  a 
degree,  application  or  transfer  research.  Health 
services  research  includes  work  done  on  tech- 
nologies or  systems  that  are  still  considered  to 


be  in  the  development,  transfer,  and  application 
stages,  as  well  as  on  technologies  or  systems 
that  are  in  widespread  use.  Actually,  the  lines  of 
definition  in  terms  of  where  biomedical  research 
activities  end  and  where  health  services  research 
begins  are  rather  blurred.  This  is  because  it  is 
rare  for  a  technology  or  innovation  to  proceed 
in  a  linear  process  from  basic  research  to  wide- 
spread application.  It  is  also  unlikely  that  a 
single  agency  or  other  institution  will  have  an 
innovation  under  its  purview  for  the  full  range 
of  developmental  needs  that  are  part  of  produc- 
ing a  usable  end  product. 

In  the  discussion  that  follows,  the  National 
Institutes  of  Health  (NIH)  is  used  to  represent 
the  biomedical  research  process.  The  National 
Center  for  Health  Services  Research  (NCHSR) 
and  the  Health  Care  Financing  Administration 
(HCFA)  are  the  examples  of  Federal  agencies 
that  sponsor  health  services  research.  And  the 
National  Center  for  Health  Care  Technology 
(NCHCT)  is  the  example  of  an  agency  whose  ac- 
tivities incorporate  or  relate  to  both  types  of 
R&D. 


BIOMEDICAL  RESEARCH  — NIH 

NIH  is  not  the  only  Federal  agency  to  conduct 
or  fund  biomedical  research.  The  Veterans  Ad- 
ministration, the  Office  of  Naval  Research,  the 
National  Science  Foundation  (NSF),  the  Depart- 
ment of  Energy,  and  others  are  involved  to 
varying  degrees  in  a  range  of  biomedical  re- 
search activities.  NIH,  however,  is  by  far  the 
largest  single  provider  of  biomedical  research 
funds  in  the  United  States.  NIH  covers  a  wide 
range  of  scientific  activity  and  uses  peer  review, 
as  well  as  program  and  project  evaluation,  proc- 
esses that  are  similar  to  those  of  other  Federal 
agencies  that  support  health  care  R&D. 

NIH  is  an  agency  of  the  Public  Health  Service 
(PHS)  in  the  Department  of  Health  and  Human 
Services  (DHHS).  Its  mandate,  stated  in  broad 
terms,  is  to  improve  human  health  by  increasing 
understanding  of  the  processes  underlying 
health  and  acquiring  new  knowledge  to  prevent, 
detect,  diagnose,  and  treat  disease  and  disabili- 
ty. 


NIH  pursues  this  mission  via  an  array  of 
intramural  programs  conducted  at  NIH  and 
through  an  extensive  network  of  extramural 
grants  and  contracts  to  private  and  public  insti- 
tutions in  the  United  States  and  other  countries. 
Its  budget  in  1980  will  be  approximately  $3.4 
billion,  which  represents  approximately  68  per- 
cent of  the  Federal  obligations  for  health  R&D. 
Forty-one  percent  of  total  national  health  R&D 
support  (Federal  and  State  Government,  indus- 
try, and  private  nonprofit  organizations),  is 
provided  by  NIH  (1978  estimates)  (446). 

NIH  is  organized  into  11  institutes,  the  Na- 
tional Library  of  Medicine  (NLM),  and  6  re- 
search and  support  divisions.  Two  of  the  insti- 
tutes (the  National  Cancer  Institute  (NCI)  and 
the  National  Heart,  Lung,  and  Blood  Institute 
(NHLBI)),  as  well  as  NLM,  have  "bureau"  sta- 
tus; the  other  institutes  are  "division"  level  orga- 
nizations. These  various  semiautonomous  orga- 
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nizations  are  coordinated  through  the  Office  of 
the  Director  of  NIH. 

There  are  several  levels  of  control  and  input 
in  the  resource  allocation  process  for  biomedical 
research.1  Every  year,  NIH  is  subjected  to  nu- 
merous examinations  of  its  allocation  of  re- 
search support,  its  selection  and  implementation 
of  research  priorities,  and  its  requests  for  funds 
for  the  upcoming  year(s).  This  process  extends 
from  congressional  hearings  on  NIH  budget  au- 
thorization and  appropriation  levels  to  the 
study  section  advisory  groups  that  meet  three 
times  each  year  to  evaluate  the  technical  and/ or 
scientific  merit  of  research  proposals.  Further- 
more, it  is  not  unusual  for  an  ad  hoc  presidential 
or  congressional  panel,  commission,  or  task 
force  to  express  its  opinions  and  conclusions  re- 
garding the  quality,  quantity,  or  usefulness  of 
NIH's  efforts.  These  "advice-giving"  groups  can 
carry  considerable  weight  in  the  priority-setting 
and  allocation  process.  Another  group  that  can 
affect  the  amount  of  support  given  a  particular 
research  area  are  the  scientists  themselves.  The 
thousands  of  scientists  who  continue  to  initiate 
and  support  various  types  of  research,  submit 
grant  and  contract  proposals,  and  remain  in  an 
area  of  research  for  extended  periods  of  time  can 
have  considerable  bearing  on  national  research 
priorities. 

Congress  plays,  and  has  played,  a  significant 
role  in  the  creation,  expansion,  and  contraction 
of  research  efforts  at  NIH.  Its  budget-setting  and 
oversight  authority  are  powerful  levers  in  the 
decisionmaking  and  allocational  process. 
Strickland  (586,587),  Ward  (612),  and  others 
(91,367)  have  noted  that  medical  research  is  a 
national  policy  issue  that  entails  all  the  political 
pushing  and  shoving  that  is  characteristic  of 
other  national  allocational  issues.  The  alloca- 
tion of  funds  for  medical  research  is  neither  just 
a  budgeting  exercise  nor  a  purely  scientific  deci- 
sion that  is,  or  will  be,  made  only  by  the  scien- 
tific community.  In  part,  this  is  because  medical 


'NIH  is  frequently  praised  for  its  contribution  to  the  fields  of  sci- 
ence and  medicine,  and  its  achievements  in  these  areas  have  been 
described  as  extraordinary.  Yet  in  recent  years,  the  charge  has 
been  leveled  that  its  accomplishments  have  not  been  great  enough 
given  the  large  sums  of  money  that  have  been  channeled  its  way.  It 
is  beyond  the  intent  or  capabilities  of  this  study  to  enter  this  de- 
bate. 


research,  technological  advances,  new  treat- 
ments and  cures,  and  the  health  care  research 
system  itself  have  come  under  the  scrutiny  of  the 
general  public.  Reader's  Digest,  Ann  Landers,2 
Marcus  Welby,  M.D.,  disease-oriented  interest 
groups,  and  others  have  turned  millions  of 
Americans  into  supporters  or  critics  of  various 
aspects  of  biomedical  research,  in  particular, 
and  the  health  care  system,  in  general. 

Congress  is  in  the  position,  both  constitution- 
ally and  politically,  to  exercise  a  great  deal  of 
control  over  the  priority-setting  and  allocation 
process  at  NIH  through  its  "power  of  the  purse." 
Maldonado  sums  up  congressional  influence  via 
"the  appropriating  art"  (379): 

Health  budget  review  and  development  fall 
under  the  jurisdiction  of  the  subcommittees  on 
Labor-HEW  (of  the  House  and  Senate  commit- 
tees on  appropriations).  The  subcommittee  on 
Labor-HEW  has  authority*  to  (1)  recommend 
the  appropriations  or  funding  level  in  support  of 
program  or  research  activities  (or  their  termina- 
tion); (2)  through  the  report  vehicle,  earmark 
funds  for  specific  programs  or  projects,  set  pro- 
gram directions,  instruct,  warn,  and  exhort;  (3) 
accept  or  reject  proposed  impoundments  (rescis- 
sions and  deferrals);  (4)  approve  or  disapprove 
transfer  of  unexpended  balances;  and  (5)  con- 
duct studies  and  examinations  of  agency/de- 
partment operations  and  organizations. 

The  appropriations  subcommittees  have  tradi- 
tionally played  a  prominent  role  in  health  policy 
by  "earmarking"  funds  for  specific  activities, 
and  "requesting"  or  "expecting"  that  a  certain 
"emphasis"  or  direction  be  taken.**  In  sum- 
mary, they  set  program  direction,  emphasis,  and 
budget  levels  for  health. 

A  number  of  other  forces  operate  both  within 
and  outside  Government  to  ensure  that  a  given 
health  care  issue  receives  attention  in  the  deci- 


2  An  example  of  the  tremendous  response  that  can  be  generated 
by  "popular"  authors,  TV  shows,  publications,  etc.,  is  given  by 
Robert  Q.  Marston  regarding  the  debate  between  those  who  did 
not  want  a  "cancer  crusade,"  and  those  who  did.  He  cites  the  fact 
Ann  Landers  stimulated  more  than  a  million  responses  to  a  state- 
ment in  one  of  her  columns  in  1971  asking  for  support  of  the  can- 
cer crusade  (382). 

*W.  H.  Brown,  Rules  of  the  House  of  Representatives,  pp. 
349-351,  393,  396-399,  Washington,  D.C.:  U.S.  Government 
Printing  Office,  1975  (71). 

**R.  F.  Fenno,  The  Power  of  the  Purse:  Appropriations  Politics 
in  Congress,  Little,  Brown,  Boston,  1966.  (199). 
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sionmaking  process.  The  executive  branch  can 
marshall  a  considerable  collection  of  expertise — 
from  NIH,  the  Food  and  Drug  Administration 
(FDA),  HCFA,  special  task  forces  of  experts, 
and  numerous  advisory  councils  that  are  part  of 
the  R&D  process — to  provide  advice  to  the  pol- 
icy process.  The  scientific  community,  the  many 
disease-oriented  organizations  (e.g.,  the  Ameri- 
can Cancer  Society,  the  Cystic  Fibrosis  Founda- 
tion) and  professional  organizations  (e.g.,  the 
American  Medical  Association,  the  American 
Hospital  Association)  provide  a  mix  of  voices 
that  add  to  the  diversity  of  views  on  various 
health  care  issues.  At  any  time,  on  any  given 
health  care  issue,  there  are  likely  to  be  coalitions, 
within  both  the  legislative  and  executive 
branches  of  Government  that  have  the  support 
or  opposition  of  the  many  nongovernmental  in- 
terest groups — all  urging  special  consideration 
for  their  programs  on  the  research  agenda.  For  a 
detailed  description  of  this  process,  see  Rettig's 
(510)  and  Strickland's  (587)  accounts  of  the 
"war  on  cancer"  declared  in  1971  by  President 
Nixon. 

Rarely  are  formal  decision-assisting  tech- 
niques, especially  CEA/CBA,  explicitly  used  to 
make  decisions  at  this  broad  political  or  societal 
level  of  the  policy  process.  Although  the  preced- 
ing discussion  has  been  based  on  NIH,  much  of 
what  was  said  about  decisions  at  this  policy  lev- 
el pertains  to  the  health  services  research  sys- 
tem, as  well. 

Peer  Review  Allocation  and 
Evaluation  Mechanisms 

NIH  and  each  institute  within  NIH  must  de- 
cide how  to  divide  available  resources  among: 
1)  extramural  grants,  2)  contract  research,  and 
3)  intramural  projects  initiated  by  scientists 
within  NIH.  In  the  case  of  extramural  grants, 
further  consideration  must  be  given  to  the 
allocation  of  resources  for  investigator-initiated 
research  grants  and  large,  complex  multidisci- 
plinary  research  team  efforts  such  as  center  and 
program  project  grants.  To  some  extent,  NIH 
priority-setting  and  research  selection  is  based 
on  the  relative  merits  of  basic,  applied,  or  trans- 
fer research  in  each  institute's  area  and  the 
budget  the  institute  will  receive. 


The  mechanism  evolved  at  NIH  to  manage 
these  many  considerations  is  a  peer  review  sys- 
tem. Most,  if  not  all,  of  the  Federal  agencies  that 
support  health  care  research  rely  on  some  form 
of  peer  review  to  solicit  expert  opinion  regard- 
ing the  potential  success  of  a  proposed  project. 
The  peer  review  system  of  NIH  consists  of  2,200 
primarily  non-Federal  scientists  and  lay  advi- 
sors from  across  the  Nation.  These  individuals 
are  grouped  into  130  peer  review  groups,  advi- 
sory committees,  councils,  and  panels  (449), 
whose  function  is  to  provide  NIH  with  expert 
opinions  both  on  the  scientific  and  technical 
merit  of  grant  applications  and  contract  propos- 
als and  also  on  program  initiatives  and  policy 
issues. 

Extramural  Grants 

The  peer  review  system  for  grant  applications 
used  by  NIH  is  based  on  two  sequential  levels  of 
review,  referred  to  as  the  "dual  review  system." 
The  first  level  involves  panels  of  experts  estab- 
lished according  to  scientific  disciplines  or  cur- 
rent research  areas  for  the  primary  purpose  of 
evaluating  the  scientific  and  technical  merit  of 
grant  applications.  In  the  Division  of  Research 
Grants  (DRG),  discussed  below,  the  panels  are 
called  study  sections.  Generally,  however,  the 
panels  are  referred  to  as  initial  review  groups 
(IRGs). 

The  second  level  of  review  is  by  a  national  ad- 
visory council  or  board,  referred  to  here  as  a 
"council."  Council  recommendations  are  based 
not  only  on  considerations  of  scientific  merit  as 
judged  by  IRGs,  but  also  on  the  relevance  of  a 
grant  application  to  an  institute's  programs  and 
priorities. 

Receipt  and  Assignment  of  Applications. — 

Grant  applications  submitted  to  NIH  are  re- 
ceived centrally  in  DRG.  This  Division,  one  of 
the  research  and  support  divisions  at  NIH,  is  not 
connected  to  the  research  institutes  or  responsi- 
ble for  funding  or  managing  grant  programs. 

DRG  screens  incoming  grant  applications,  de- 
termines the  relevance  of  each  application  to  the 
overall  mission  of  NIH,  and  assigns  acceptable 
applications  to  an  appropriate  IRG  and  to  an 
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appropriate  institute.3  Assignment  to  an  IRG  is 
based  on  the  complementarity  of  a  proposed  re- 
search project  to  the  review  responsibilities  and 
scientific  expertise  of  IRG's  members;  assign- 
ment to  an  institute  is  based  on  the  institute's 
legislatively  mandated  program  responsibility. 
If  the  subject  matter  of  an  application  is  perti- 
nent to  the  program  responsibilities  of  two  insti- 
tutes, a  dual  assignment  may  be  made.  Should 
the  primary  institute  decide  not  to  provide  fund- 
ing, the  other  institute  may  consider  the  applica- 
tion for  funding. 

Initial  Review  by  Peers. — Depending  on  the 
type  of  research  proposed,  the  first  level  of  sci- 
entific and  technical  merit  review  is  by  an  IRG 
located  either  within  DRG  or  within  an  insti- 
tute. IRGs  in  the  institutes  are  usually  multidis- 
ciplinary  and  are  thus  constituted  to  review 
more  complex  program  project  and  center  grant 
applications.  An  NIH  health  scientist  adminis- 
trator serves  as  executive  secretary  of  each  re- 
view group. 

IRG  members,  who  serve  up  to  4  years  per 
appointment,  meet  three  times  a  year  to  review 
applications.  When  assessing  the  scientific  and 
technical  merit  of  an  application  assigned  to 
their  IRG,  the  members  consider,  among  other 
criteria:  the  importance  of  the  proposed  re- 
search problems;  the  originality  of  the  ap- 
proach; the  training,  experience,  and  research 
competence  or  promise  of  the  investigators;  the 
adequacy  of  the  experimental  design;  the  suit- 
ability of  the  facilities;  and  the  appropriateness 
of  the  requested  budget  to  the  work  proposed 
(164). 

For  each  application,  IRG  makes  a  recom- 
mendation of  approval,  disapproval,  or  deferral 
for  additional  information  by  majority  vote.  In 
addition,  for  each  application  recommended  for 
approval,  each  member  of  IRG  individually  and 
privately  records  a  numerical  rating  that  reflects 
a  personal  evaluation  of  the  scientific  merit  of 
the  proposed  research  or  training.  The  numer- 
ical rating  is  from  1.0  (the  most  meritorious)  to 


The  Referral  Branch  also  receives  and  assigns  applications  to 
other  agencies  within  PHS,  such  as  the  Alcohol,  Drug  Abuse,  and 
Mental  Health  Administration,  the  Center  for  Disease  Control,  the 
Health  Services  Administration,  the  Health  Resources  Administra- 
tion, and  the  Food  and  Drug  Administration. 


5.0  (the  least  meritorious),  with  0.1  increments. 
After  the  meeting,  the  executive  secretary  aver- 
ages the  individual  reviewers'  ratings  for  each 
approved  application  and  multiplies  this  by  100 
to  provide  a  three-digit  number  known  as  the 
priority  score.  Priority  scores  assist  the  staff  of 
the  institutes  in  determining  which  applications 
are  to  be  funded. 

If  information  is  needed  that  is  not  in  the  ap- 
plication and  cannot  be  obtained  by  telephone 
or  mail,  a  project  site  visit  may  be  made  either 
prior  to  an  IRG  meeting  or  after  an  IRG  deferral 
recommendation.  In  addition,  site  visits  are 
often  routinely  made  when  an  application  in- 
volves complex  coordination.  In  either  case,  the 
executive  secretary  assembles  a  team  of  site 
visitors.  For  a  research  project  grant  applica- 
tion, the  site  visit  team  generally  includes  two  or 
more  members  of  IRG,  the  executive  secretary, 
a  member  of  the  institute  staff,  and  usually  one 
or  more  ad  hoc  consultants  who  are  experts  in 
critical  aspects  of  the  proposed  work.  For  more 
complex  grant  applications  (e.g.,  those  for  pro- 
gram project  or  center  grants),  the  site  visit  team 
may  include  as  many  as  10  to  15  individuals,  in- 
cluding members  of  IRG  and  ad  hoc  consult- 
ants. 

After  IRG  meets,  the  executive  secretary  pre- 
pares a  summary  statement  for  each  application 
and  forwards  it  to  the  appropriate  institute  for 
review  by  its  council.  The  summary  statement 
contains  a  description  and  critique  of  the  pro- 
posed activity,  an  explanation  of  the  recommen- 
dation of  IRG,  a  recommended  budget,  and  no- 
tations about  any  special  points  (e.g.,  a  split 
vote  or  a  potentially  hazardous  experimental 
procedure). 

Review  by  National  Advisory  Councils  or 
Boards. — Each  NIH  funding  unit  has  a  council 
that  must  review  and  recommend  action  on  the 
applications  received  from  IRGs.  These  councils 
are  mandated  by  law,  and  some  have  minimum 
levels  placed  both  on  the  number  of  times  they 
must  meet  each  year  and  on  the  number  of 
members  they  must  have.4  Members  include  au- 

"For  example,  the  National  Cancer  Advisory  Board  was  estab- 
lished in  August  of  1937  by  Public  Law  244,  75th  Cong.,  and  has 
since  been  renamed  and  restructured  by  subsequent  legislation  that 
required  that  the  board  shall  meet  "not  less  than  four  times  a  year" 
(449). 
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thorities  in  scientific  and  health  fields  directly 
related  to  the  program  interests  of  the  institute, 
as  well  as  lay  people  noted  for  their  interest  or 
activity  in  national  health  problems.  With  the 
exception  of  individual  fellowship  applications 
and  some  grant  applications  recommended  at 
yearly  levels  not  exceeding  $35,000  (in  NCI  and 
NHLBI),  grants  cannot  be  awarded  without  ap- 
proval by  a  council. 

The  councils  review  research  proposals  in  a 
broader  context  than  IRGs,  because  the  coun- 
cils' recommendations  are  based  not  only  on 
IRG  scientific  and  technical  merit  evaluations, 
but  also  on  the  needs  of  NIH  and  the  missions  of 
the  individual  institutes,  the  need  for  initiation 
of  research  in  new  areas,  the  degree  of  relevance 
of  the  proposed  research  to  the  missions  of  the 
institutes,  and  other  policy  issues  (164).  In  addi- 
tion, a  major  focus  of  the  councils  is  the  SATT5 
criteria  that  have  been  developed  to  guide  and 
categorize  the  support  for  biomedical  research. 

The  councils  forward  their  recommendations 
for  each  application  to  the  institute  director  for 
a  decision  on  funding.  Usually,  the  approved 
projects  are  chosen  according  to  rank  until  the 
budget  is  obligated.  An  approved  grant  applica- 
tion is  not  assured  of  funding,  because  there  are 
almost  always  more  eligible  applications  than 
available  funds.  An  unapproved  application, 
however,  cannot  be  funded  (164). 

R&D  Contracts 

Contracts  are  used  to  procure  a  specified  re- 
search product  or  service  from  a  nonprofit  or  a 
commercial  organization.  The  initiative  for 
these  contracts  generally  comes  from  NIH  pro- 
gram staff  and  advisors,  who  identify  a  specific 
research  or  service  need.  Workshops  and  con- 
ferences are  a  source  of  invaluable  ideas  and 
guidance. 


5S  stands  for  Science  Base  activities.  This  category  is,  for  all 
practical  purposes,  basic  research  support.  A  is  clinical  Appli- 
cation, research  focused  on  intervention.  It  is  at  this  level  that  clin- 
ical trials  take  place.  T  represents  the  Transfer  research  that  NIH 
undertakes  to  move  products  or  interventions  that  have  survived 
testing  to  the  consumers  and  providers  of  health  care.  Demonstra- 
tions, practical  trials,  and  consensus  development  conferences  are 
part  of  the  transfer  process.  The  final  T  stands  for  the  Training 
function  that  NIH  supports.  This  effort  is  geared  to  supporting  and 
attracting  people  into  the  field  of  research. 


Each  institute  has  developed  slightly  different 
methods  and  procedures  for  using  contracts  to 
satisfy  its  research  needs.  The  basic  mechanisms 
the  various  institutes  use  to  develop  contract 
proposals,  to  review  contract  applications,  and 
to  evaluate  the  progress  and  outcomes  of  con- 
tract products,  though,  are  similar  enough  to  be 
summarized  in  a  general  description. 

The  scientific  staff  members  within  a  given  in- 
stitute, with  assistance  from  standing  commit- 
tees or  ad  hoc  advisory  groups,  develop  a  re- 
search project  description  and  plan.  In  compli- 
ance with  the  law  that  mandates  peer  review  for 
NIH  contract  projects,  the  concept  of  the  project 
is  evaluated  by  a  scientific  review  group  com- 
posed largely  of  non-Federal  advisors.  The  pro- 
posed project  is  then  released  as  an  RFP  (request 
for  proposal),  which  specifies  the  terms,  condi- 
tions, and  provisions  for  the  requested  contract. 
RFPs  are  announced  in  the  Commerce  Business 
Daily  as  required  by  law,  in  the  NIH  Guide  for 
Grants  and  Contracts  as  required  by  policy,  and 
in  other  appropriate  publications  as  determined 
by  program  requirements. 

In  response  to  an  RFP,  applicants  submit  con- 
tract proposals,  which  are  reviewed  by  the  in- 
stitute's contracting  officer  and  then  by  a  scien- 
tific review  group  consisting  mainly  of  non- 
Government  scientists  chosen  for  their  expertise 
in  the  relevant  area.  Their  recommendations  are 
sent  to  a  contract  review  committee  composed 
of  senior  program  staff  from  the  funding  insti- 
tute. Applicants  determined  to  be  in  the  "com- 
petitive range"  have  an  opportunity  to  further 
defend  or  clarify  their  proposals  via  written  or 
oral  discussion  with  the  contracting  officer  and 
senior  program  staff.  Once  the  applicants  have 
made  their  "best  and  final"  offer,  the  remaining 
applications  are  reevaluated  via  further  negotia- 
tions in  order  to  determine  the  one  to  be  funded. 

In  addition  to  contracts  solicited  by  NIH,  un- 
solicited contract  proposals  are  also  occasional- 
ly received  by  DRG,  and  then  sent  to  the  appro- 
priate institutes.  If  the  unsolicited  proposal  is 
relevant  to  the  institute's  program  needs,  it  is  re- 
viewed by  the  contracting  officer  and  scientific 
review  group  in  a  process  similar  to  that  for  so- 
licited proposals. 
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The  progress  and  products  of  contract  re- 
search are  under  the  supervision  and  review  of  a 
project  officer  at  the  funding  unit.  Informal,  as 
well  as  formal,  procedures  are  used  to  monitor 
the  performance  of  the  contract  project.  A  ma- 
jor difference  between  contract  research  and 
grant  and  intramural  research,  at  least  theoreti- 
cally, is  that  contractors  are  required  to  provide 
an  end  product  based  on  specifications  estab- 
lished by  the  funding  unit  before  the  research 
begins.  The  other  forms  of  research  support  are 
usually  not  as  tightly  bound  by  requirements  to 
produce  a  given  outcome  at  the  end  of  their  re- 
search. 


Intramural  Research 

Intramural  research  projects  are  developed, 
supported,  and  evaluated  by  a  mixture  of  insti- 
tute staff  and  outside  advisors.  Each  institute 
has  an  in-house  scientific  board  that  meets  on  a 
regular  basis  to  set  institute  policy,  review  in- 
stitute programs,  and  discuss  research  needs. 
For  each  institute,  the  scientific  board  also  iden- 
tifies future  goals,  needs,  and  capabilities  for  in- 
tramural research. 

Intramural  research  ideas  or  project  proposals 
can  be  initiated  by  the  scientific  board,  by  indi- 
vidual researchers,  laboratory  chiefs,  or  by  sci- 
entific directors.  Research  proposals  are  dis- 
cussed by  in-house  scientific  staff,  the  institute's 
scientific  board,  and  outside  experts  if  needed. 
Depending  on  the  available  space,  personnel, 
and  budget,  and  the  feedback  from  this  informal 
peer  review  process  conducted  in-house,  a  spe- 
cific project  is  started,  rejected,  or  deferred. 

Additional  aspects  of  the  system  are  publica- 
tion of  research  in  journals  and  presentation  of 
work  to  in-house  staff  and  the  scientific  commu- 
nity. Directors  from  each  institute  meet  each 
month  to  review  the  work  of  selected  nonten- 
ured  researchers  who  are  candidates  for  tenured 
status  at  NIH.  Finally,  each  institute  has  a  Board 
of  Scientific  Counselors,  composed  of  non-Gov- 
ernment scientists,  that  meets  twice  each  year  to 
review  intramural  projects  and  programs  for 
scientific  performance  and  progress. 


Discussion  of  Peer  Review  Mechanisms 

The  peer  review  mechanisms  at  NIH,  by  and 
large,  appear  to  have  worked  quite  well  over  the 
last  several  decades.  This  is  not  to  say  that  the 
peer  review  system  has  been  beyond  criticism  or 
change.  There  have  been  a  score  of  studies, 
hearings,  reports,  and  reviews  on  the  peer  re- 
view process  at  NIH  over  the  last  25  years,  and 
these  many  assessments  have  led  to  revalua- 
tions and  modifications  of  the  procedures  used 
to  conduct  the  research  support  processes.6  De- 
spite these  changes,  however,  the  fundamental 
concept  and  framework  of  the  peer  review  sys- 
tem at  NIH  remain  intact.  Furthermore,  the  peer 
review  system  remains  the  mechanism  of  choice 
for  efficiently  and  effectively  allocating  research 
resources. 

The  research  and  resource  allocation  process, 
both  in  biomedical  and  health  services  research, 
is  an  uncertain  endeavor  that  entails  probability 
and  risk.  The  peer  review  process  is  an  attempt 
at  predicting  the  probability  of  success  for  a 
given  scientific  effort.  Through  this  process, 
NIH  and  the  other  research-supporting  agencies 
attempt  to  estimate  or  predict  a  subjective  level 
of  quality  performance  of  a  researcher,  given 
the  very  real  confines  of  a  budget,  a  limited  time 
frame,  the  existing  knowledge  and  technological 
base,  the  availability  of  trained  researchers  will- 
ing and  able  to  work  on  a  given  problem,  re- 
search space  and  resources,  and  the  presence  of 
often  conflicting  and  changing  health  care  goals 
and  policies.  In  a  very  real  sense,  the  peer  re- 
view process  is  an  attempt,  either  explicit  or  im- 
plied, to  select  and  support  "cost-effective"  re- 
search. 


6Since  1947,  there  have  been  approximately  18  official  reviews, 
studies,  inquiries,  hearings,  and  reports  examining  the  adequacy  of 
the  peer  review  system  at  NIH  (447).  The  most  recent  study  to 
assess  the  mettle  of  the  peer  review  system  was  supervised  by  Dr. 
Ruth  L.  Kirchstein,  Director  of  the  National  Institute  of  General 
Medical  Sciences.  The  study  team  received  its  go  ahead  in  April  of 
1975  and  submitted  phase  I  of  the  report  to  the  Director  of  NIH  in 
December  of  1976,  and  the  final  report,  phase  II,  in  December  of 
1978  (445,447,448).  For  additional  views  on  the  peer  review  sys- 
tem, see  Report  of  the  President's  Biomedical  Research  Panel  (491); 
Opinions  on  the  NIH  Grants  Peer  Review  System,  Phase  II  of  the 
Report  to  the  Director  (447);  Investigation  of  the  National  Insti- 
tutes of  Health  (115);  and  "Support  of  New  Principal  Investigators 
by  NIH:  1966-1972"  (153). 
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Performance  Evaluations 

Even  though  NIH  very  rarely  uses  or  funds 
CEA/CBAs  in  either  the  allocation  or  evalua- 
tion process  (see  survey  of  CEA/CBA  use  at 
NIH  in  app.  B),  the  various  institutes  do  employ 
a  number  of  analytical  techniques  to  assess  the 
performance  of  the  research  process.  Most  of 
the  techniques  are  employed  after  a  "product" 
has  been  turned  in.  It  is  important  to  note,  how- 
ever, that  allocation  and  evaluation  efforts  at 
NIH  often  merge  into  the  same  function.  In  the 
minds  of  most  NIH  administrators  and  program 
and  project  leaders,  evaluation  activities  are 
ongoing  efforts  that  cannot  be  clearly  or  neatly 
separated  into  the  categories  of  planning,  allo- 
cation, and  oversight.  A  number  of  the  tech- 
niques that  NIH,  other  research-supporting 
agencies,  and  analysts  in  general  use  to  evaluate 
the  performance  of  research  efforts  are  de- 
scribed in  the  following  paragraphs. 

General  Asessments  of  Biomedical  Research 

Citation  Analysis. — A  form  of  performance 
evaluation  that  has  gained  considerable  atten- 
tion over  the  last  decade  is  the  use  of  citation 
analysis.  This  analytical  technique  examines  the 
quantity  and,  to  a  degree,  the  quality  of  scien- 
tific papers,  reports,  articles,  and  so  on,  that  re- 
sult from  research  projects.  NIH,  NSF,  and 
others  have  used  this  technique  to  evaluate  the 
performance  of  their  research  efforts.  NIH  and 
NSF  have  supported  a  number  of  studies  over 
the  last  5  years  that  tested:  1)  the  correlation  be- 
tween their  support  efforts  and  biomedical  pub- 
lication output  (225,425);  2)  the  extramural  role 
of  NIH  as  a  research  support  agency  (426);  3) 
the  relationship  between  the  peer  review  system, 
citations,  and  biomedical  research  policy  (86); 
and  4)  other  aspects  of  research  support  (565). 
Comroe  and  Dripps  (120)  and  the  Department 
of  Defense  (565)  have  employed  a  variation  of 
citation  analysis  to  evaluate  the  development  of 
specific  scientific  and  technological  innovations. 

Morbidity /Mortality  Studies. — Another  type 
of  performance  evaluation  is  the  attempt  to 
measure  the  reductions  in  mortality  and/ or 
morbidity  that  have  occurred  as  a  result  of  bio- 
medical research  and  health  care  in  general. 
Morbidity/mortality  studies  attempt  to  examine 


the  achievements  of  biomedical  research  and  the 
health  care  system  that  are  responsible  for  pro- 
longing life,  improving  quality  of  life  and  avert- 
ing health  care  costs,  as  well  as  a  host  of  other 
averted  costs  and  added  benefits  that  have  re- 
sulted from  the  investment  in  biomedical  R&D 
(9,421,422,512). 

In  essence,  both  techniques  above  examine 
the  usefulness  of  a  field  of  science  over  the 
course  of  a  number  of  years.  For  the  immediate 
needs  of  institute,  program,  or  project  manag- 
ers, more  specific  decision-assisting  techniques 
are  required. 

Evaluations  of  Specific  Technologies 
and  Programs /Projects 

To  assess  the  performance  of  specific  medical 
technologies  and  programs  and  projects  that  it 
administers,  NIH  rarely  uses  explicit  cost-effec- 
tiveness or  cost-benefit  studies,  but  it  does  use  a 
variety  of  effectiveness  studies.  For  this  report, 
the  effectiveness  studies  will  be  grouped  into 
two  categories:  1)  clinical  trials  and  consensus 
development  conferences,  which  are  used  to 
evaluate  the  safety  and  efficacy  of  specific  tech- 
nologies; and  2)  more  general  performance  eval- 
uations, which  are  used  to  examine  a  diversity 
of  NIH  programs  and  projects.  Whereas  clinical 
trials  deal  primarily  with  medical  technologies 
that  are  in  the  development/application  stage  of 
research,  performance  evaluations  can  cover  a 
myriad  of  types  of  NIH  activity  ranging  from 
biomedical  research  to  health  services  research. 
These  evaluations  are  used  for  obtaining  infor- 
mation necessary  for  program  and  administra- 
tive needs. 

Clinical  Trials  and  Consensus  Development 
Conferences. — Clinical  trials  provide  the  basis 
for  the  testing,  evaluation,  and  application  of 
basic  and  applied  research  knowledge  before  it 
is  introduced  into  the  health  care  system.  They 
also  provide  the  information  needed  to  examine 
the  safety  and  efficacy  of  newly  emerging  tech- 
nologies. NIH  is  the  major  source  of  support  for 
these  trials  in  the  Federal  Government  (465). 
Clinical  trials  are  technical  in  nature  and  are  not 
usually  designed  to  examine  in  depth  any  fac- 
tors other  than  the  safety  and  efficacy  of  a  medi- 
cal technology. 
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Augmenting  the  clinical  trial  process  are  the 
recently  implemented  consensus  development 
conferences  that  NIH  sponsors.  These  confer- 
ences are  designed  to  go  a  step  beyond  the  lim- 
ited focus  of  clinical  trials  to  assess  a  broader  set 
of  issues.  A  sample  of  the  issues  discussed  at  re- 
cent meetings  are  methods  of  diagnosing  and 
treating  allergies,  treatments  of  ocular  mela- 
noma, and  estrogen  use  in  postmenopausal 
women.  These  meetings  may  examine  the  scien- 
tific merit,  along  with  the  attending  issues  of 
risks,  benefits,  costs,  and  ethical  implications, 
of  implementing  a  new  medical  technology.  The 
meetings  are  primarily  a  technically  oriented  ap- 
proach to  discussing  whether  the  innovation  is 
safe,  efficacious,  and  cost  effective.7 

Project  Assessments. — The  second  type  of 
evaluation  is  the  much  broader  area  of  project 
assessment  that  NIH  uses  to  examine  completed 
and  ongoing  projects.  All  11  institutes,  NLM, 
and  the  Division  of  Research  Resources  support 
evaluation  efforts  within  their  areas  of  responsi- 
bility. 

Evaluation  activities  are  divided  into  three 
general  categories  (450):  1)  the  NIH  evaluation 
classifications,  such  as  program  effectiveness, 
methodology  and  resource  development;  2) 
funding  guidelines,  i.e.,  science  base,  appli- 
cation, (technology)  transfer,  and  training 
(SATT);  and  3)  the  Assistant  Secretary  for  Pro- 
gram Evaluation  (ASPE)  program  evaluation 
categories. 

Four  of  the  ASPE  evaluation  categories  will 
be  mentioned  (450): 

•  exploratory  evaluations,  which  identify  the 
objectives  and  expectations  of  relevant  pol- 
icymakers and  program  managers,  identify 
the  program  objectives  and  performance 
indicators  on  which  the  program  will  be 
held  accountable,  and  identify  evaluation 
management  options  for  changing  program 
activities,  objectives,  or  uses  of  informa- 
tion in  ways  that  are  likely  to  improve  pro- 
gram performance; 

•  short-term  evaluations,  which  summarize 
available  and  readily  obtainable  informa- 

7For  a  more  detailed  discussion  of  clinical  trials  and  consensus 
development  panels,  see  the  OTA  report  Asessing  the  Efficacy  and 
Safety  of  Medical  Technologies  (465). 


tion  on  program  performance  in  terms  of 
the  objectives  and  performance  indicators 
identified  in  the  exploratory  evaluation  and 
provide  designs  for  full-scale  evaluations; 

•  full-scale  evaluations  of  program  perform- 
ance in  terms  of  the  agreed-upon  measur- 
able objectives  and  performance  indicators 
identified  in  the  exploratory  evaluations; 
and 

•  program  performance  summaries,  which 
summarize  evidence  on  how  programs  are 
performing  in  terms  of  the  set  of  objectives 
and  performance  indicators  on  which  the 
program  is  being  held  accountable. 

The  projects  that  fall  into  these  four  cate- 
gories, in  that  they  attempt  to  examine  explicit 
outputs  and/or  the  inputs  of  a  given  project,  are 
the  closest  NIH  comes  to  conducting  explicit 
cost-effectiveness  studies.  The  boundaries  estab- 
lished in  the  definitions  above  are  not  much  dif- 
ferent from  the  general  outlines  used  in  many 
cases  for  CEAs.  The  studies'  main  divergence 
from  CEAs  is  in  their  relatively  weaker  empha- 
sis on  "costs"  and  stronger  focus  on  the  per- 
formance, or  effectiveness,  aspects  of  the  pro- 
gram or  project.  Cost  comparisons  remain  im- 
plicit; the  quality  of  project  performance,  how- 
ever, receives  considerable  attention. 

Potential  Use  of  CEA/CBA 

The  formal  technique  of  CEA/CBA  seems  ill 
suited  to  the  NIH  biomedical  research  decision- 
making process  (560).  Perhaps  this  methodolo- 
gy could  be  helpful  in  certain  program  or  proj- 
ect evaluation  situations,  or  possibly  in  the  con- 
text of  assisting  the  awarding  of  contracts  or 
supporting  the  research  centers,  but  generally 
the  complex,  dynamic,  and  uncertain  nature  of 
the  research  process,  the  frailty  of  the  methodol- 
ogy, and  the  backgrounds  of  the  decisionmakers 
militate  against  the  beneficial  use  of  these  tech- 
niques in  the  biomedical  research  system. 

The  uncertainties  involved  in  the  biomedical 
R&D  process  are  many,  and  CEA/CBA  has  lit- 
tle ability  to  adequately  summarize,  include, 
and  compare  items  involved  in  bringing  re- 
search to  fruition.  The  development  of  a  medi- 
cal technology  does  not  follow  a  linear  or  steady 
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path.  Biomedical  science  may  contribute  only  a 
portion  of  the  knowledge  and  research  that  is 
needed  to  develop  an  idea  or  technology  fully. 
Even  after  the  technology  is  in  use,  the  obstacles 
to  defining  and  measuring  the  costs,  effective- 
ness, and  outcomes  are  many.  Distributional 
and  equity  issues  present  themselves  at  many 
points  along  the  R&D  process.  These  method- 
ological problems  have  yet  to  be  solved.  The  list 
could  be  continued,  but  the  heart  of  the  issue  is 
that  formal  CEA/CBA  is  not  readily  useful  or 
applicable  to  the  process  of  planning,  allocat- 
ing, or  evaluating  biomedical  research. 


The  case  against  CEA  and  CBA  grows  even 
stronger  when  one  examines  the  mechanisms 
already  in  place  to  assist  the  biomedical  R&D 
decisionmaking  process.  Those  mechanisms,  the 
peer  review  system  and  evaluation  processes, 
seem  to  have  performed  adequately  over  the 
years  to  allocate  research  resources  efficiently 
and  intelligently.  As  the  rough  edges  are  re- 
moved from  those  systems,  they  become  even 
more  valuable  to  the  decisionmaking  process. 
At  the  level  of  biomedical  R&D,  a  cost-effective- 
ness attitude  probably  serves  the  system  better 
than  would  formal  CEA/CBA. 


NATIONAL  CENTER  FOR  HEALTH  CARE  TECHNOLOGY 


In  1978,  Congress  added  a  new  level  of 
evaluation  and  coordination  to  the  health  care 
research,  development,  and  application  proc- 
ess by  establishing  NCHCT  as  part  of  HEW.8 
NCHCT  is  responsible  to  the  Assistant  Secre- 
tary of  Health,  DHHS.  Its  mandate  is  to  "under- 
take and  support  assessments  of  health  care 
technologies." 

Before  the  creation  of  NCHCT,  there  was  no 
identifiable  organization  that  could  act  as  a 
coordinator  for  information  concerning  emerg- 
ing technologies.  There  was  no  single  office  that 
had  the  responsibility  to  act  as  the  information 
manager  for  the  application  and  dissemination 
of  new  medical  technologies.9 

NCHCT  has  a  potentially  important  role  in 
the  decisionmaking  process.  Its  enabling  legisla- 
tion establishes  a  number  of  broad-ranging 
functions  for  NCHCT.  Generally,  NCHCT  is  to 
set  priorities  for  technology  assessment  and  to 
encourage,  conduct,  and  support  assessments, 
research,  demonstrations,  and  evaluations  con- 
cerning health  care  technology.  Specifically,  the 
Center  will  (437): 


"The  statute  establishing  NCHCT  is  Public  Law  95-623,  sec.  309 
of  the  Health  Services  Research,  Health  Statistics,  and  Health  Care 
Technology  Act,  Nov.  9,  1978. 

'With  respect  to  new  medical  technologies,  the  responsibility 
and  involvement  of  NIH  tends  not  to  extend  much  past  the  applied 
research  and  early  transfer  stages.  The  health  services  research  sec- 
tor usually  does  not  focus  on  new  technologies  until  they  are  in 
place  or  at  least  well  along  the  development  cycle. 


•  undertake  and  support  comprehensive  as- 
sessments of  health  care  technology,  in- 
cluding analyses  of  safety  and  efficacy,  and 
ethical  issues; 

•  undertake  and  support  studies  of  the  cost 
effectiveness  and  cost/benefit  of  current 
and  developing  technologies; 

•  undertake  and  support  syntheses  of  existing 
research  (e.g.,  state-of-the-art  papers); 

•  provide  the  best  scientific /medical  and  eco- 
nomic assessments  to  HCFA  on  medicare 
coverage  for  specific  medical  procedures 
and  technologies,  including  evaluation  of 
the  costs  and  benefits  of  old  procedures  and 
assessment  of  new  technologies  for  which 
HCFA  might  require  medicare  coverage  de- 
cisions in  the  future; 

•  undertake  and  support  dissemination  of  in- 
formation derived  from  its  assessment  ac- 
tivities to  the  practicing  and  scientific  com- 
munities, Federal  agencies  with  health  in- 
terests, third-party  payers,  the  public,  and 
others  as  appropriate; 

•  undertake  and  support  manpower  training 
programs  to  provide  for  an  expanded  and 
continuing  supply  of  individuals  qualified 
to  perform  the  research,  demonstration, 
and  evaluation  activities  related  to  health 
care  technology;  and 

•  undertake  and  support,  to  the  extent  prac- 
ticable, by  September  1,  1981,  the  plan- 
ning, establishment,  and  operation  of  three 
extramural  centers  for  assessments,  re- 
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search,  demonstrations,  and  evaluations  of 
issues  in  health. 

By  law,  NCHCT  must  have  a  national  council 
to  advise  the  NCHCT  staff.  A  major  function 
the  council,  which  has  been  established,  will 
serve  is  the  identification  and  selection  of  med- 
ical technologies  that  should  receive  priority  at- 
tention.10 The  council  is  also  asked  to  (142): 

•  advise  the  Secretary  on  the  safety,  efficacy, 
effectiveness,  cost  effectiveness,  and  social, 
ethical,  and  economic  implications  of  par- 
ticular health  care  technologies; 

•  develop,  publish,  and  disseminate  stand- 
ards, norms,  and  criteria  concerning  the 
use  of  particular  technologies,  when  appro- 
priate and  practicable;  and 

•  review  applications  for  grants  and  con- 
tracts exceeding  $35,000  in  direct  costs. 

NCHCT  is  less  than  2  years  old.  Much  of  its 
activity  since  its  creation  has  been  directed 
towards  organizing  and  developing  procedures 
to  perform  the  functions  it  was  created  to  serve. 
For  that  reason,  it  is  difficult  to  examine 
NCHCT  in  the  same  light  as  NIH,  NCHSR,  and 
HCFA — agencies  that,  together,  have  several 
decades  of  experience  behind  them. 

Research  Support  Structure 

The  research  support  structure  of  NCHCT  is 
similar  to  that  of  NIH,  NCHSR,  and  HCFA. 
NCHCT  will  support  research  via  extramural 
grants,  extramural  contracts,  intramural  re- 
search, and  will  support  manpower  training 
programs  to  draw  qualified  individuals  into  this 
area  of  research.  The  national  council  is  respon- 
sible for  reviewing  any  grant  exceeeding  $35,000 
in  direct  costs,  and  outside  experts  can  be  used 
to  review  and  provide  comment  on  any  applica- 
tions for  research  funds  or  any  results  of  re- 
search that  are  received  by  NCHCT.  The  peer 
review  system  and  the  basic  organizational  pro- 


10The  NCHCT  National  Council  identified  the  following  tech- 
nologies as  candidates  for  priority  consideration  (457): 

Ultrasonic  diagnostic  procedures:  coronary  by-pass  surgery;  fetal 
monitoring:  end-stage  renal  disease — dialysis  (home  v.  hosp.),  trans- 
plantation, etc.;  non-invasive  radioactive  imaging;  barium  enema; 
EEG:  nuclear  magnetic  resonance;  auto-analyzers;  all  skull  films;  cere- 
bral angiography;  dental  X-rays:  CAT  scanner;  continuous  flow  anal- 
ysis. 


cedures  for  selecting  and  supporting  research  at 
NCHCT  are  much  the  same  as  elsewhere. 

Technology  Evaluation  Activities 

NCHCT  is  in  the  process  of  pursuing  a  range 
of  research  and  dissemination  activities  and  is  in 
the  planning  stages  for  several  additional  proj- 
ects to  be  initiated  this  year  (1980).  At  the  first 
national  council  meeting  in  October  of  1979, 
Ruth  Hanft,  Acting  Deputy  Assistant  Secretary 
for  Health  Research,  Statistics,  and  Technol- 
ogy, reviewed  a  number  of  the  activities  that  are 
ongoing  or  in  the  planning  stages  at  the  Center. 
These  activities  are  (142): 

1.  Comprehensive  assessment. — Safety,  effi- 
cacy, cost  effectiveness,  and  economic, 
social,  and  ethical  impact  of  a  selected 
technology  will  comprise  a  comprehensive 
assessment.  Two  such  assessments  will  be 
initiated  in  fiscal  year  1980. 11 

2.  Coverage  issues. — HCFA  asks  the  Center 
for  advice  regarding  the  appropriateness 
of  paying  for  the  use  of  certain  technolo- 
gies with  medicare  funds.  Currently,  the 
Center  is  responding  to  53  requests  for 
coverage  recommendations. 

3.  Consensus  development  processes  of  NIH. 
— The  Center  will  be  more  involved  in  the 
consensus  development  conferences  spon- 
sored by  NIH  and  will  cosponsor  some  of 
these.  NCHCT  provided  an  economic 
analysis  at  the  conference  on  intraocular 
lenses  conducted  in  September  1979. 

4.  Intramural  activities. — The  Center  is  con- 
ducting cost-effectiveness  studies  on  a 
number  of  technologies  (e.g.,  intraocular 
lenses,  estrogen  use  by  postmenopausal 
women,  and  antenatal  diagnosis). 

5.  Overviews. — The  Center  is  writing  state- 
of-the-art  papers  on  technologies  which 
are  candidates  for  comprehensive  assess- 


llIn  the  fall  of  1979,  NCHCT  was  handed,  or  chose,  its  first  tech- 
nology assessment  topic — the  alpha  fetoprotein  test  kits.  FDA  is 
delaying  approval  of  this  new  technology  that  it  has  found  to  be 
safe  and  effective  until  the  Center  completes  an  assessment  of  the 
kit's  impact  on  the  health  care  system  and  society.  NCHCT  is  co- 
ordinating the  information,  input,  and  efforts  of  IDA,  the  Center 
for  Disease  Control,  NIH,  the  Health  Resources  Administration, 
HCFA,  and  others  in  order  to  evaluate  the  full  range  of  issues  in- 
volved. 
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ments  (e.g.,  end-stage  renal  disease,  elec- 
troencephalography, and  coronary  by- 
pass surgery). 

6.  Dissemination. — This  activity  is  just  be- 
ginning, with  the  assistance  of  NIH  and 
FDA  among  others.  In  fiscal  year  1980, 
the  Center  will  begin  its  own  dissemina- 
tion activities. 

7.  Early  warning  system. — HCFA,  the  Cen- 
ter for  Disease  Control,  FDA,  NIH,  and 
the  Alcohol,  Drug  Abuse,  and  Mental 
Health  Administration  are  developing 
methodologies  to  identify  emerging  tech- 
nologies. Non-Federal  organizations,  such 
as  the  American  Hospital  Association,  • 
will  also  help  identify  emerging  technolo- 
gies. 

8.  Grant  and  contract  program. — This  pro- 
gram began  in  fiscal  year  1980  in  the  area 
of  literature  syntheses,  cost-effectiveness 
studies,  and  economic,  social,  and  ethical 
analyses. 

9.  Centers  program. — Public  Law  95-623  re- 
quires that  three  extramural  research  cen- 
ters be  established  by  September  1,  1981. 

As  this  list  of  activities  indicates,  NCHCT  is 
involved  in  a  wide  range  of  technology  evalua- 
tion efforts.  At  one  end  of  the  technology  eval- 
uation process,  the  Center  is  involved  in  consen- 
sus development  activities  at  NIH  which  are  fo- 
cusing on  relatively  new  medical  technologies 
that  have  areas  of  uncertainty  to  be  resolved.  In 
addition,  a  very  important  part  of  the  Center's 
efforts  is  focused  on  the  reimbursement  system 
(see  ch.  5).  NCHCT  is  specifically  charged  with 
coordinating  information  and  making  recom- 
mendations to  HCFA  regarding  new  or  existing 
medical  technologies.  All  indications  are  that 
this  will  be  a  priority  activity  of  the  Center. 
Finally,  the  Center  will  provide  comprehensive 
examinations  of  medical  technologies  that  have 
been  in  use  for  some  time.  In  many  of  its  func- 
tions, NCHCT  has  the  authority  to  conduct  or 
fund  CEAs. 

Potential  Use  of  CEA/CBA 

NCHCT  is  in  a  position  to  act  as  an  informa- 
tion broker  to  a  number  of  agencies  at  a  variety 
of  decision  points  in  the  policymaking  process. 


Its  mandate  is  so  broad,  however,  that  it  may 
become  overwhelmed  by  the  number  and  diver- 
sity of  functions  it  is  asked  to  perform.  Funding 
levels,  and  consequently  staffing  levels,  are 
significantly  lower  than  those  called  for  in 
NCHCT's  authorizing  legislation.12  This  factor 
may  significantly  affect  the  number  and  range 
of  duties  the  Center  can  be  expected  to  perform. 
So  far,  HCFA  has  requested  NCHCT  to  exam- 
ine 53  coverage  issues.  With  this  area  of  respon- 
sibility and  the  other  functions  listed  above, 
NCHCT  will  likely  find  it  difficult,  at  current 
funding  and  staff  levels,  to  totally  fulfill  the  ex- 
pectations placed  on  it. 

NCHCT  has  developed  a  priority-setting 
process  that  may  help  it  handle  the  influx  of  re- 
quests for  information,  recommendations,  as- 
sessments, and  general  assistance.  To  a  degree, 
priority-setting  is  vested  in  the  national  council, 
but  the  council  focuses  primarily  on  the  selec- 
tion of  medical  technologies  that  warrant  full- 
scale  assessments.  The  remainder  of  the  prior- 
ity-setting process  is  a  mixture  of  identifying  in- 
house  needs,  perceptions  of  what  areas  require 
more  immediate  action,  the  assimilation  of 
priority  areas  and  views  of  the  agencies  NCHCT 
must  work  with  and  respond  to,  and  the  general 
consensus  of  the  scientific  and  medical  commu- 
nity regarding  issues  that  require  attention.  In 
addition,  NCHCT  priorities  will  be  set  to  a  de- 
gree by  the  deadlines  of  the  agencies  with  which 
NCHCT  works. 

The  result  of  these  many  factors  and  consider- 
ations is  an  unclear  picture  of  how  formal  CEA/ 
CBA  fits  into  NCHCT's  activities.  The  agency  is 
new;  it  remains  to  be  seen  to  what  extent 
NCHCT  will  conduct  or  support  formal  CEA/ 
CBA.  If,  where,  and  when  these  techniques  are 
employed,  it  will  be  interesting  to  note  their 
level  of  sophistication,  their  use  in  decisionmak- 
ing efforts,  and  their  level  of  input  into  and  im- 
pact upon  the  policy  process. 


uThe  enabling  legislation  (Public  Law  95-623,  sec.  309)  author- 
ized $15  million,  $25  million,  and  $35  million  for  fiscal  years  1979 
through  1981,  respectively,  for  NCHCT  to  carry  out  its  mandate. 
The  fiscal  year  1979  budget  for  the  Center  was  $344,000,  and  the 
fiscal  year  1980  budget  is  $3.25  million. 
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HEALTH  SERVICES  RESEARCH 

Health  services  research  is  relatively  new  as 
an  organized  field  of  research.  Only  in  the  last 
two  decades  has  there  been  significant  Federal 
involvement  in  this  field.  Before  Federal  sup- 
port, health  services  research  efforts  were  spo- 
radic and  infrequent.  The  infusion  of  Federal  in- 
volvement and  financial  support  helped  to  orga- 
nize the  disparate  parts  into  a  more  focused  field 
of  inquiry. 

The  activities  represented  by  the  National 
Center  for  Health  Statistics  (see  app.  B)  have  a 
history  going  back  at  least  to  1946  when  Con- 
gress first  authorized  vital  statistics  activities. 
Two  of  the  Nation's  primary  centers  for  guid- 
ing, conducting,  funding,  and  disseminating 
health  services  research  efforts — NCHSR  and 
HCFA — are  12  and  3  years  old,  respectively.13 
HCFA  tends  to  focus  most  of  its  research  on  its 
program  needs.  NCHSR,  however,  was  created 
to  conduct  and  support  broadly  based  health 
services  research  that  does  not  necessarily  have 
to  be  tied  to  any  agency's  program  needs.  Since 
NCHSR's  arrival  on  the  health  services  research 
scene,  several  Federal  organizations  have  been 
authorized  to  pursue,  or  have  assumed,  varying 
degrees  of  responsibility  for  health  services 
research  activities.14 

The  past  decade  of  federally  supported  health 
services  research  efforts  has  raised  numerous 
questions  regarding  the  uses  and  usefulness  of 
health  services  research,  its  "cost  effectiveness," 
and  its  impact  on  the  decisionmaking  process.  A 
great  deal  of  discussion  has  centered  on  the  fac- 
tors and  mechanisms  needed  to  address  these 
questions  and  improve  health  services  research. 
Analysts  who  have  examined  many  of  these  is- 
sues have  often  reached  diametrically  opposed 


"Although  HCFA  is  3  years  old,  the  parts  that  comprise  the 
Agency  have  existed  for  many  years.  The  medicare  and  medicaid 
programs  and  the  Office  of  Research  and  Statistics  were  originally 
located  in  the  Social  Security  Administration.  The  PSRO  program 
at  HCFA  was  preceded  by  the  Health  Standards  and  Quality  Bu- 
reau in  the  Office  of  the  Assistant  Secretary  of  Health  at  HEW. 

14An  Institute  of  Medicine  report  identifies  at  least  15  Federal  de- 
partments, agencies,  or  commissions,  that  conduct  or  fund  some 
form  of  health  services  research.  The  Federal  involvement  ranges 
from  that  of  the  Federal  Trade  Commission  to  that  of  the  National 
Aeronautics  and  Space  Administration  and  DHHS  (formerly 
HEW)  (428). 


conclusions  regarding  the  performance  of  serv- 
ices research.  One  side  of  the  debate  cites  nu- 
merous examples  of  major  health  policy  deci- 
sions where  services  research  was  of  little  conse- 
quence (31,359,383,423).  The  other  side  cites 
evidence  to  support  the  notion  that  health  serv- 
ices research  has  been  successful  in  producing 
information  useful  to  the  policy  process,  in 
some  instances  has  been  instrumental  in  stimu- 
lating policy  debate  and  change,  and  in  general, 
compares  favorably  to  other  forms  of  social 
services  research  (139,186,428,566). 15 

It  is  doubtful  whether  the  debate  surrounding 
the  performance  and  usefulness  of  health  serv- 
ices research  will  be  settled  conclusively  one 
way  or  the  other.  The  range  and  types  of  issues 
that  services  research  must  address,  the  dynam- 
ic nature  of  the  policy  issues  and  health  care  sys- 
tem, and  the  complexity  of  the  decisionmaking 
process  combine  to  almost  preclude  a  conclu- 
sion regarding  the  relative  worth  of  the  informa- 
tion produced  by  health  services  research  ef- 
forts. Nevertheless,  there  does  seem  to  be  a  gen- 
eral feeling  that  health  services  research  has  not 
attained  the  high  level  of  expectations  set  for  it  a 
decade  ago  and  that  improvements  can  be 
made.  The  ongoing  examination  and  evaluation 
of  services  research  may  help  improve  its  focus, 
usefulness,  and  quality. 

The  primary  focus  of  this  section  is  on  the 
current  or  potential  use  of  CEA/CBA  in  the 
health  services  research  decisionmaking  proc- 
ess, i.e.,  the  procedures  used  to  establish  health 
services  research  priorities,  fund  research  proj- 
ects, and  evaluate  work  that  is  ongoing  or  that 
has  been  completed.  The  section  also  discusses 
the  extent  to  which  certain  health  services  re- 
search agencies  support  CEA/CBAs  as  part  of 
their  research  missions. 

NCHSR  and  HCFA  represent  the  Federal 
Government's  most  substantial  commitment  to 
the  area  of  health  services  research.  NCHSR  and 


15It  is  beyond  the  intent  of  this  assessment  to  examine  this  de- 
bate. For  detailed  discussions  of  the  history  of  services  research,  its 
contributions,  its  weaknesses,  and  its  development,  see  the  follow- 
ing references  listed  at  the  end  of  this  report:  24,  36,  210,  428,  and 
643. 
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HCFA  provide  the  highest  level  of  funding  of 
health  services  research  in  the  United  States. 
The  funding  levels  for  health  services  research 
have  declined  in  actual  and  real  terms  since  the 
peak  years  of  the  early  1970's.  NCHSR's  1978 
budget  represented  less  than  40  percent  of  its 
purchasing  power  for  research  and  training  pro- 
grams compared  to  the  levels  of  the  early  1970's 
(428).  Together,  NCHSR  and  HCFA  contribute 
roughly  40  percent  of  the  total  amount  of  Feder- 
al funds  allocated  to  health  services  research 
(428).  In  fiscal  year  1980,  they  will  control 
approximately  $50  million  in  moneys  ear- 
marked for  services  research.  These  agencies  oc- 
cupy a  very  influential  position  in  the  health, 
services  research  community  and  are  in  a  posi- 
tion to  exercise  considerable  influence  on  the 
content,  direction,  and  level  of  health  services 
research  in  this  country. 

It  is  unclear  where  applied  biomedical  re- 
search ends  and  where  health  services  research 
begins.  Several  agencies  that  focus  primarily  on 
conducting  and  supporting  basic  and  appied  re- 
search routinely  conduct  or  fund  health  services 
research  as  part  of  their  programmatic  missions 
(428). 

Health  services  research  is  unlike  most  other 
areas  of  scientific  inquiry  in  that  it  is  not  orga- 
nized around  a  single  discipline  with  unique  per- 
spectives, closely  drawn  areas  of  expertise,  com- 
mon methodologies  or  techniques,  and  standard 
nomenclatures.  Health  services  research  is  a 
mixture  of  concepts,  methodologies,  attitudes, 
and  professions  that  could  easily  span  a  large 
university's  graduate  school  catalog.  The  field 
of  health  services  research  must  accommodate 
data,  methodological  frameworks,  disciplines, 
and  perspectives  from  the  diverse  fields  of  medi- 
cine, other  health-care-related  disciplines  (epi- 
demiology, nursing,  public  health,  etc.),  bio- 
statisticians,  engineers,  lawyers,  demographers, 
geographers,  operation  researchers,  economists, 
social  workers,  hospital  and  business  adminis- 
trators, and  so  forth.  Individual  health  services 
researchers  tend  to  approach  the  issues  from  the 
confines  and  perspective  of  their  particular 
discipline. 

David  Mechanic  describes  the  purpose  of 
health  services  research  as  follows  (396): 


The  health  services  research  field  focuses  on 
the  production,  organization,  distribution,  and 
impact  of  services  on  health  status,  illness,  and 
disability  ...  it  concentrates  attention  on  im- 
proving the  distribution,  quality,  effectiveness, 
and  efficiency  of  medical  care. 

A  study  by  the  Institute  of  Medicine  on  health 
services  research  attempted  to  provide  guide- 
lines for  the  description  or  classification  of  the 
types  of  health  services  research  that  exist.  Ac- 
cording to  that  report,  a  study  is  classified  as 
health  services  research  if  it  satisfies  two  criteria 
(428): 

1.  It  deals  with  some  features  of  the  struc- 
ture, processes,  or  effects  of  personal 
health  services. 

2.  At  least  one  of  the  features  is  related  to  a 
conceptual  framework  other  than  that  of 
contemporary  applied  biomedical  science. 

National  Center  for 
Health  Services  Research 

NCHSR  was  created  in  1968  without  explicit 
congressional  authorization.  It  was  not  until 
1974  that  NCHSR  received  legislative  authority 
via  the  Health  Services  Research,  Health  Statis- 
tics, and  Medical  Libraries  Act  (Public  Law 
93-353).  Since  then,  several  laws  have  added  to 
or  modified  NCHSR's  research  domain.16 

NCHSR  has  two  principal  responsibilities. 
One  is  to  develop  information  that  might  be 
used  by  various  decisionmakers  in  the  public 
and  private  sectors.  The  other  is  to  ensure  that 
the  information  that  results  from  the  research, 
evaluation,  and  demonstration  activities  of 
NCHSR  is  disseminated  rapidly  and  in  a  form 
that  is  usable. 

NCHSR  is  a  major  supporter  of  broadly  fo- 
cused health  services  research.  Its  fiscal  year 
1980  operating  budget  will  be  almost  $30  mil- 
lion. NCHSR  is  almost  unique  in  the  Federal 
Government  in  that  it  can  sponsor  health  serv- 
ices research  apart  from  direct  administrative  or 


16A  few  of  those  laws  are  the  National  Health  Planning  and  Re- 
sources Development  Act  of  1974  (Public  Law  93-641),  the  Emer- 
gency Medical  Services  Systems  Act  of  1973  (Public  Law  93-154, 
sec,  1205),  and  the  Health  Maintenance  Organizations  Act  of 
1973.  These  and  others  have  influenced  the  direction  of  research 
priorities  and  level  of  funding  from  NCHSR. 
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programmatic  needs.  It  is  not  responsible  for  the 
administration  of  any  health  care  delivery  or  re- 
imbursement activities;  it  exists  solely  to  con- 
duct and  sponsor  health  services  research  and  to 
disseminate  the  results  of  that  research  to  rele- 
vant Government  agencies,  the  research  com- 
munity, and  other  interested  parties. 

The  NCHSR  statute's  language  is  so  broad 
that  the  Agency  retains  considerable  leeway  in 
its  selection  of  specific  research  issues  to  pursue. 
To  identify  the  areas  of  research  it  needs  to  con- 
duct and  support,  NCHSR  employs  an  informal 
consensus  development  process. 

Priority  Setting 

The  priority-setting  process  involves  several 
steps.  First,  NCHSR  canvasses  policymakers, 
consumers,  Government  and  non-Government 
experts,  health  care  providers,  professional  as- 
sociations, program  administrators,  and  others, 
in  an  attempt  to  identify  current  and  emerging 
issues  that  present  the  most  immediate  problems 
to  the  health  care  system.  In  the  first  cut  at  the 
list  of  suggested  issues,  a  number  of  criteria  are 
used,  two  of  which  are  the  relative  importance 
of  the  issue  and  the  perception  that  there  is  a 
good  chance  that  the  research  will  provide  in- 
formation that  will  contribute  substantively  to 
the  policy  process. 

NCHSR  staff,  as  well  as  outside  professionals 
involved  in  various  research  areas,  are  involved 
in  the  culling  process.  The  Director  of  NCHSR 
selects  the  top  priority  concerns  from  the  major 
issues  identified  by  f^ys  process.  Before  the  list  is 
adopted,  it  is  submnted  to  an  assorted  group  of 
NCHSR  members  and  non-NCHSR  experts  for 
review.  The  priority  issues  that  emerge  from 
this  process  become  the  areas  of  health  services 
research  that  NCHSR  focuses  on.  The  issue  se- 
lection process  is  not  a  one-time  occurrence,  but 
rather  an  ongoing  interaction  between  NCHSR 
and  the  health  care  community. 

The  list  of  priority  areas  does  not  lock  out  all 
other  health  care  issues.  NCHSR  can,  and  does, 
consider  "meritorious  and  potentially  impor- 
tant" proposals  for  a  range  of  nonpriority  is- 
sues. In-house,  or  intramural,  research  can  also 
pursue  areas  of  interest  that  fall  outside  the  pri- 
ority areas. 


Currently,  NCHSR  supports  five  general  pri- 
ority areas  and  a  special  studies  category  that  is 
concerned  with  examining  issues  of  emerging 
importance  or  of  research  methodology.  The 
current  priority  areas  are  (438):  1)  health  care 
costs  and  cost  containment,  2)  health  insurance, 
3)  planning  and  regulation,  4)  technology  and 
computer  science  applications,  and  5)  health 
manpower 

Evaluation  of  Research  Proposals 

NCHSR's  health  services  research  efforts  are 
allocated  among  intramural  research,  extra- 
mural research  grants  and  contracts,  and  center 
grants.  Intramural  research  is  subject  to  an  in- 
formal review  process  within  NCHSR.  When 
the  need  arises,  staff  proposals  for  in-house 
research  projects  may  also  be  reviewed  by  non- 
Federal  experts.  Projects  are  formulated  accord- 
ing to  special  research  needs  or  personnel  capa- 
bilities at  NCHSR.  The  in-house  project  pro- 
posal moves  through  the  administrative  hier- 
archy, receives  comments  and  suggestions  from 
the  staff,  and  may  be  circulated  to  outside  re- 
viewers before  final  action  is  taken  by  NCHSR. 
Health  services  research  centers  receive  funding 
apart  from  the  extramural  and  intramural  re- 
search budget.  NCHSR's  authorizing  legislation 
called  for  the  funding  of  at  least  six  "existing  and 
new  centers  for  multidisciplinary  health  services 
research,  evaluation,  and  demonstrations." 
There  are  eight  such  centers  currently  in  opera- 
tion (650). 

NCHSR  uses  a  dual  review  type  of  evaluation 
process  to  screen  and  select  grant  and  contract 
applications  it  receives  (385).  This  process  is 
quite  similiar  to  the  peer  review  system  de- 
scribed earlier  for  NIH.  Grant  and  contract  pro- 
posals that  exceed  $35,000  in  direct  costs  are  re- 
quired to  be  reviewed  for  scientific  and  technical 
merit  by  study  sections  composed  of  non-Feder- 
al Government  experts.  Proposals  that  do  not 
exceed  $35,000  are  evaluated  by  NCHSR  staff, 
and  where  needed  outside  reviewers. 

Explicit  CEA/CBAs  are  not  used  in  the  prior- 
ity-setting, project  selection,  or  research  evalua- 
tion processes.  A  few  of  the  project  criteria  used 
in  the  selection  process,  however,  make  it  likely 
that  there  will  be  significant  emphasis  on  the 
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relationship  between  the  application's  budget 
(cost)  and  its  potential  outcomes  (benefits). 
NCHSR  receives  far  more  applications  than  it 
has  the  funds  to  support,  so  in  the  project  selec- 
tion process,  it  does  consider  cost  and  effec- 
tiveness. The  budget  and  outcomes  criteria  used 
to  evaluate  applications  are  considered  with  ref- 
erence to  the  Agency's  budget  and  goals. 

CEA/CBA  as  Part  of  NCHSR's 
Research  Mission 

NCHSR  supports  a  number  of  cost-effective- 
ness and  cost-benefit  studies  that  cover  a  broad 
range  of  health  services  issues  (439)  and  is  the 
major  supporter  of  CEAs  and  CBAs  in  the  Fe'd- 
eral  health  care  research  system  at  this  time. 
Two  of  NCHSR's  five  research  priority  areas — 
health  care  costs  and  cost  containment,  and 
planning  and  regulation — specifically  call  for  re- 
search using  CEAs  to  examine  the  issues  within 
these  priority  areas  (438). 

The  CEAs  and  CBAs  that  NCHSR  supports 
range  from  rigorous  analyses  of  specific  health 
care  topics  to  broad  studies  of  more  complex 
health  care  issues.  The  results  of  these  studies, 
like  most  of  NCHSR's  research  products,  are 
circulated  via  the  Agency's  formal  publications 
dissemination  process,  conferences,  seminars, 
journals,  announcements,  etc.  NCHSR  is  in  a 
position  to  monitor  the  eventual  use  of  the  re- 
search results  by  other  agencies  and  decision- 
makers. In  most  cases,  though,  the  nature  of  the 
policymaking  process  makes  it  quite  difficult  to 
determine  to  what  extent  a  given  piece  of  in- 
formation is  used  to  reach  a  final  policy  deci- 
sion. NCHSR  is  powerless  to  assure  that  the  re- 
search results  will  actually  be  used  in  the  deci- 
sionmaking process.  The  Agency's  legislative 
mandate  is  to  support  health  services  research 
projects  that  answer,  or  at  least  address,  the 
issues  that  the  research  community  and  policy- 
makers feel  are  important.  Decisions  about 
whether  and,  if  so,  how  to  use  the  information 
that  is  generated  are  left  up  to  agencies  and  deci- 
sionmakers outside  NCHSR. 

Health  Care  Financing  Administration 

HCFA,  established  in  1977  as  a  result  of  a  ma- 
jor reorganization  at  the  Department  of  Health, 


Education,  and  Welfare  (now  DHHS),  is  a  re- 
cent addition  to  the  Federal  Government's 
health  services  research  community.  This  Agen- 
cy is  the  organizational  center  that  administers 
medicare/medicaid  programs,  the  Professional 
Standards  Review  Organization  (PSRO)  pro- 
gram, and  the  research  and  statistics  programs 
for  these  areas. 

The  legislation  authorizing  these  programs  al- 
lows HCFA  to  consider  and  pursue  a  range  of 
health  services  research  topics.17  HCFA  is  par- 
ticularly responsible  for  sponsoring  research 
that  relates  to  its  primary  mission:  the  adminis- 
tration and  evaluation  of  the  medicare/medic- 
aid area  and  the  PSRO  function.  The  mandate 
of  the  Office  of  Policy  Planning  and  Research  at 
HCFA,  however,  is  broad  enough  to  include  a 
wider  range  of  research  topics.18 

HCFA  currently  has  five  major  priority  areas 
of  grant  support.  Specifically,  Agency  support 
is  focused  on  projects  that  meet  one  of  the  fol- 
lowing criteria  (284): 

1.  develops  or  demonstrates  new  financing 
mechanisms  for  health  care  services; 

2.  utilizes  financing  mechanisms  to  influence 
the  effectiveness  or  delivery  of  health  care 
services; 

3.  develops  or  demonstrates  management  or 
administrative  procedures  that  will  benefit 
HCFA  programs; 

4.  develops  knowledge  or  undertakes  anal- 
yses of  the  basic  nature  and  structure  of 
health  care  costs  and  factors  that  affect 
their  rate  of  increase;  or 

5.  examines  the  economic  and  behavioral 
relationships  between  the  financing  of 
health  care  services  and  the  total  activities 
of  the  health  care  sector  of  the  economy. 


17The  legislation  that  establishes  HCFA's  scope  of  responsibility 
is  the  Social  Security  Amendments  of  1972  (Public  Law  92-603). 
Sections  222  and  245  of  this  law  provide  for  health  services  re- 
search and  demonstration  projects  for  a  variety  of  reimbursement 
issues,  performance  incentives,  etc.  The  National  Health  Planning 
and  Resources  Development  Act  of  1974  (Public  Law  93-641) 
added  provisions  for  work  in  ratesetting  and  development  of  qual- 
ity data  for  providers. 

"The  Office  of  Policy  Planning  and  Research  of  HCFA  grew  out 
of  the  Office  of  Research  and  Statistics  (ORS),  formerly  of  the  So- 
cial Security  Administration.  ORS  became  a  major  source  of  sup- 
port for  health  services  research  in  the  late  1930's. 


Ch.  9— R&D  Programs  •  111 


Priority  Setting 

HCFA's  priority  research  areas  are  chosen  in 
an  informal  process  much  like  the  one  used  by 
NCHSR.  Opinions,  suggestions,  and  recom- 
mendations are  solicited  from  a  variety  of  peo- 
ple with  a  broad  background  of  experience  and 
training.  Their  recommendations  are  culled  to 
form  a  list  of  research  priorities.  Although  pri- 
ority areas  receive  special  attention  in  the  selec- 
tion process,  new  ideas  and  innovations  are  not 
automatically  excluded  from  consideration  if 
they  fall  outside  these  areas.  Relevant  examples 
of  the  issues  on  which  HCFA  is  focusing  are  hos- 
pital costs,  physician  reimbursement,  and  the 
quality  and  effectiveness  of  various  health  care 
areas. 

Evaluation  of  Research  Proposals 

The  peer  review  mechanism  and  the  project 
evaluation  process  at  HCFA  are  very  similiar  to 
the  systems  at  NCHSR.  The  major  difference  be- 
tween the  two  approaches  is  the  formal  involve- 
ment of  HCFA  staff  in  the  initial  technical  re- 
view panels.  Instead  of  being  composed  of  all 
non-Federal  experts  like  NCHSR's  and  NIH's 
panels,  HCFA's  review  panels  are  composed  of 
an  equal  mixture  of  HCFA  staff,  non-Federal 
experts,  and  non-HCFA  Government  experts 
(284).  The  criteria  used  to  evaluate  the  merit  of 
the  research  application  are  geared  as  much  to 
HCFA's  program  needs  as  to  technical  and  sci- 
entific merit.  Final  review  and  funding  decisions 
are  made  by  HCFA  staff  and  administrators. 

Use  of  CEA/CBA 

HCFA  is  like  the  other  agencies  discussed 
above,  in  that  it  does  not  use  explicit  CEA/CBA 
to  select  research  goals  or  grant  proposals,  or  to 
evaluate  project  outcomes.  Awareness  of  proj- 
ects' "cost  effectiveness"  to  the  Agency's  goals, 
however,  is  present. 

HCFA  focuses  much  of  its  substantive  re- 
search on  priority  issues  that  try  to  determine 
the  relative  efficiency  of  various  methods  of 
delivering  care.  Many  of  its  research  solicitation 
areas  are  for  work  to  arrive  at  CEA/CBA-like 


evaluations  of  specific  issues.  Or,  data  are  pur- 
sued that  might  assist  HCFA  in  determining  cost 
and  effectiveness  measures.  Several  of  the  proj- 
ects recently  completed,  as  well  as  a  number  of 
ongoing  efforts,  are  directly  focused  on  cost- 
effectiveness  issues  (283). 

The  CEA/CBAs  that  HCFA  is  supporting  are 
similar  to  those  at  NCHSR,  in  that  they  range 
from  fairly  rigorous  attempts  at  examining  the 
costs  and  effectiveness  of  a  medical  technology 
to  being  closer  to  effectiveness  studies  that  in- 
clude costs  as  an  analytical  afterthought,  if  at 
all.  The  information  produced  as  a  result  of 
these  analyses  is  combined  with  the  other  infor- 
mation and  considerations  that  are  part  of 
HCFA's  program  responsibilities.  At  HCFA,  as 
at  the  agencies  previously  described,  the  sup- 
port, evaluation,  and  use  of  analyses  or  infor- 
mation in  general  is  not  usually  a  linear  process. 
It  is  extremely  difficult  to  point  to  a  given  piece 
of  information,  a  cost-effectiveness  study  for  in- 
stance, and  determine  where,  how,  or  even  if, 
the  knowledge  gained  from  a  given  research 
project  was  directly  incorporated  into  a  given 
policy  decision.  Nevertheless,  since  HCFA  can 
tailor  its  use  of  CEA/CBA  to  suit  its  special  pro- 
gram needs  and  might  then  be  able  to  implement 
that  information  within  its  organizational 
boundaries,  it  could  possibly  serve  as  an  in- 
structive example  of  the  support  and  use  of 
CEA/CBA  in  the  decisionmaking  process. 
HCFA  is  in  a  position  to  evaluate  the  impact  of 
its  actions  after  programs  or  program  changes 
have  been  in  place  for  a  period  of  time.  NCHSR 
and  NCHCT  do  not  share  this  type  of  start-to- 
finish  authority.  NCHSR  and  NCHCT  have 
more  limited  access  and  input  to  the  decision- 
making process  than  do  HCFA,  NIH,  and  other 
health  care  agencies  that  maintain  program  re- 
sponsibilities. 

Although  HCFA  is  a  collection  of  fairly  ma- 
ture programs  that  have  been  reorganized  into  a 
new  agency,  the  organization  has  not  fully  set- 
tled into  its  new  niche.  It  will  be  interesting  to 
note  how,  or  if,  HCFA  uses  any  CEA/CBAs 
that  are  produced  by  HCFA's  own  research  sup- 
port system,  by  NCHSR,  or  via  the  new  efforts 
of  NCHCT. 
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USE  OF  CEA/CBA  IN  R&D  PROGRAMS:  GENERAL  FINDINGS 


Formal  CEA/CBAs  are  not  used  or  supported 
to  a  great  extent  in  the  health  care  R&D  system. 
At  the  process  (or  administrative)  level,  Federal 
agencies  rarely  use  explicit  CEA/CBA  as  a  deci- 
sion-assisting tool.  In  only  a  few  cases  has 
CEA/CBA  been  instrumental  in  facilitating  a 
given  allocational  or  policy  decision.  (See  "Sur- 
vey of  Agency  Use  of  CEA/CBA,"  app.  B.)  The 
agencies  do  seem  to  make  efforts  to  employ  a 
"cost-effective"  approach  and  attitude  towards 
the  decisions  that  are  made  regarding  the  setting 
of  priorities,  selection  of  research  projects,  and 
in  the  evaluation  of  research  products.  These  at- 
titudes and  approaches  are  transferred  into  ac- 
tion primarily  via  external  and  internal  peer 
review  mechanisms.  At  the  substantive  level, 
several  formal  CEAs  and  CBAs  have  been  or 
will  be  supported  by  NIH,  NCHCT,  NCHSR, 
and  HCFA.  The  research  done  at  this  level  may 
feed  back  to  a  degree  into  the  administrative  de- 
cisionmaking and  evaluation  processes  at  these 
agencies.  A  significant  amount  of  the  CEA/ 
CBA  research  will  be  used  to  add  to  the  body  of 
knowledge  in  the  health  care  system. 

Efficiency-based  techniques  such  as  CEA/ 
CBA  have  not  gained  a  substantial  foothold  in 
the  R&D  decisionmaking  process.  The  same  can 
also  be  said  with  respect  to  the  other  formal 
methodologies  that  have  received  some  use  and 
recognition  in  the  field  of  policy  analysis  (i.e., 
decision  analysis,  operations  research,  technol- 
ogy assessment,  risk-benefit  analysis,  etc).  The 
relatively  infrequent  reliance  on  these  types  of 
analyses  stems  from  a  number  of  factors  related 
to  the  techniques  and  the  nature  of  the  health 
care  system  in  general  and  the  R&D  process  in 
specific.  The  complexity  and  uncertainty  of  the 
R&D  endeavor  and  its  eventual  products  does 
not  lend  itself  well  to  the  constraints  of  efficien- 
cy-based methodologies.  This  may  be  the  most 
important  reason  for  the  lack  of  use  of  CEA/ 
CBA  in  R&D.  The  health  care  R&D  process  is 
extremely  complex,  essentially  political,  and 
quite  often  is  passive.  The  Federal  R&D  process 
is  authorized,  funded,  and  supervised  by  repre- 
sentatives of  the  public  and  is  under  the  scrutiny 
of  the  Nation  as  a  whole.  The  R&D  system  must 
be  responsible  and  responsive  to  the  changing 


needs  and  goals  of  the  country;  hence,  it  is  a 
highly  political  process.  Finally,  the  system 
tends  to  be  passive.  Many  problems  are  often 
presented  to  the  decisionmaker  with  the  objec- 
tives preordained  and  the  viable  options  avail- 
able to  attain  the  goals  few  in  number.  The  R&D 
process  is  fueled  by  the  imagination  and  initia- 
tive of  those  researchers  outside  the  decision- 
making process.  A  research  goal  or  national 
health  policy  objective  can  be  established,  but 
the  system  does  not  move  without  the  initiative 
of  those  who  must  create  and  submit  research 
ideas  and  plans  to  the  funding  agencies.  To  a 
significant  degree,  R&D-supporting  agencies 
must  wait  for,  and  react  to,  the  ideas,  sugges- 
tions, efforts,  and  research  findings  of  the 
thousands  of  health  care  researchers  both  within 
and  outside  Federal  Government.  It  is  extremely 
difficult  for  CEA/CBA  to  predict,  evaluate,  or 
include  the  importance  of  the  dynamic  aspects 
of  R&D.  As  a  result,  CEA/CBA's  credibility, 
usefulness,  and  input  to  the  R&D  decisionmak- 
ing process  are  limited. 

CEA/CBA  is  most  supported  and  used  at  the 
health  services  research  end  of  the  R&D  spec- 
trum and  least  supported  and  used  at  the  basic 
and  applied  end.  This  situation  follows  logically 
from  the  inability  of  formal  analytical  tech- 
niques to  adequately  deal  with  the  high  level  of 
uncertainty  that  is  part  of  the  technology  R&D 
process.  Predictions,  opinions,  and  "guessti- 
mates" are  the  tools  of  the  trade  in  this  area. 
One  can  include  considerations  of  uncertain  fac- 
tors in  CEA/CBA;  sensitivity  analysis  can  help 
to  an  extent.  But  the  level  of  uncertainty  re- 
mains high. 

The  methodological  shortcomings  of  CEA/ 
CBA  techniques  are  compounded  by  the  atti- 
tudes of  many  of  the  potential  users  of  CEA  and 
CBA  in  the  R&D  system.  The  decisionmakers' 
perceptions  regarding  the  usefulness  and  validi- 
ty of  CEA/CBA  are  such  that  many  doubt  that 
these  techniques  are  either  necessary  or  helpful 
in  much  of  the  R&D  context.  This  view  is  par- 
ticularly strong  at  the  basic  and  applied  research 
level.  This  situation  should  not  be  surprising. 
Until  recently,  cost  containment  and  cost-effec- 
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tiveness  criteria  were  not  heavily  stressed  in 
health  care  research.  NIH  has  traditionally  fo- 
cused on  the  quality  of  the  research  supported 
and  the  safety  and  efficacy  of  the  technologies 
developed  as  results  of  research  efforts.  Finding 
cures  for  health  care  problems,  not  saving 
money,  has  been  the  primary  goal.  At  the  health 
services  end  of  the  R&D  spectrum,  distribution, 
cost,  and  quality  of  care  have  received  needed 
attention.  Cost  and  effectiveness  criteria  have 
been  part  of  the  R&D  efforts;  but  only  parts  of  a 
much  larger  focus.  Recently,  decisionmakers 
have  been  asked  to  make  them  a  larger  part  of 
their  decisionmaking  criteria.  As  a  result,  in- 
creasing numbers  of  decisionmakers  are  becom- 
ing aware  of  the  uses  (and  possible  abuses)  of 
CEA  and  CBA  techniques. 

Another  impediment  to  the  use  and  support 
of  CEA/CBA  is  the  presence  of  other  decision- 
assisting  techniques  that  have  been  fairly  suc- 
cessful in  guiding  and  informing  the  decision- 
making process.  Peer  review  panels,  publication 
of  results  in  reputable  journals,  advisory  coun- 
cils, conferences,  and  other  mechanisms  have  all 
worked  fairly  well  as  "quality"  controls  and,  to 
an  extent,  as  cost-effectiveness  filters.  These 
processes  are  firmly  in  place,  have  performed 
reasonably  well  over  the  years,  and  have  by  and 
large  produced  commendable  results.  The  pres- 
sure to  maintain  these  existing  support  systems 
might  tend  to  overwhelm  any  serious  effort  to 
incorporate  CEA/CBA  in  the  mainstream  of  the 
decisionmaking  process. 

The  organizational  and  statutory  frameworks 
are  currently  in  place  to  allow  the  use  of  CEA  or 
CBA  in  the  decisionmaking  efforts  of  the  R&D- 
supporting  agencies.  The  limiting  factors  are  the 
perceived  need  for  and  usefulness  of  the  infor- 
mation that  might  result  from  CEA/CBA  re- 
search. 

If  CEAs  could  be  adapted  to  the  need  of  bio- 
medical R&D,  the  use  of  CEA-type  studies  at 
NIH  might  logically  occur  at  the  late  transfer 
stages  of  a  technology's  movement  from  the  lab 
to  the  clinic  and  at  the  consensus  development 
meetings  held  by  NIH  and  NCHCT.  NIH  may 


also  be  able  to  incorporate  some  form  of  effi- 
ciency-based analysis  in  its  center,  contract,  and 
intramural  research  efforts.  NIH  has  more  con- 
trol over  the  formulation,  direction,  and  eval- 
uation of  these  types  of  research  efforts  than  it 
does  over  extramural  project  grant  research  ef- 
forts. It  is  at  these  points  in  the  decisionmaking 
and  information-gathering  processes  that  the 
CEA/CBA  technique  might  help  serve  the  needs 
of  the  NIH  decisionmakers.  NIH  has  in  a  very 
few  instances  used  CEA  techniques  to  evaluate 
the  ongoing  or  completed  programs  and  proj- 
ects that  are  performed  as  adjuncts  to  the  basic 
and  applied  research  missions  (e.g.,  health  edu- 
cation programs,  disease  prevention  advertising 
programs,  information  dissemination  projects, 
screening  programs,  etc.).  It  might  consider  the 
usefulness  of  increasing  its  efforts  along  these 
lines.  Once  again:  Doing  CEA/CBA  for  any 
reason  requires  that  the  limits  and  usefulness  of 
the  analyses  be  kept  in  mind. 

NCHCT  could  provide  input  into  a  range  of 
decision  points  in  the  R&D  process.  Its  legisla- 
tive mandate  authorizes  the  use  of  CEA/CBA  to 
examine  newly  emerging  and  existing  technol- 
ogies. NCHCT  cosponsors  consensus  develop- 
ment conferences  at  NIH  and  is  charged  with 
providing  information  to  HCFA  on  reimburse- 
ment issues  concerning  medical  technologies.  It 
remains  to  be  seen  how  NCHCT  will  use 
CEA/CBA  in  the  decisionmaking  process. 

NCHSR  and  HCFA  hold  promise  as  support- 
ers of  CEA/CBA  research.  HCFA  might  also  be 
a  user  of  CEA/CBA  information  in  its  PSRO 
and  medicare  programs.  These  two  agencies 
focus  on  technologies  in  later  R&D  stages  and 
on  the  services  end  of  the  health  care  system. 
CEA/CBA  has  recently  received  increased  at- 
tention in  this  area  of  research  and  one  sees  in- 
creasing use  of  this  technique  to  examine  a  host 
of  services  research  issues.19 


"Such  use  is  discussed  at  greater  length  in  Background  Paper  #2: 
Methodological  Issues  and  Literature  Review,  prepared  by  OTA  in 
conjunction  with  this  assessment. 
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INTRODUCTION  AND  BACKGROUND 


Health  maintenance  organizations  (HMOs) 
have  been  described  by  their  advocates  as  a 
"cost-effective"  way  to  provide  health  care 
(489).  It  appears  that  HMOs  do  provide  care  to 
their  enrollees  at  varied  but  substantial  cost  sav- 
ings: Empirical  evidence  shows  that  HMO  en- 
rollees pay  in  the  range  of  10  to  40  percent  less 
in  total  costs  (premium  plus  out-of-pocket  costs) 
than  conventionally  insured  comparison  groups 
(373). 1 

Explanations  of  HMO  performance,  whether 
measured  by  costs,  access,  quality,  or  physician 
or  enrollee  satisfaction,  however,  are  a  hybrid 
of  rhetoric,  theory,  and  evidence.  There  is  very 
little  information  available  on  the  details  of  how 
HMOs  actually  function,  especially  on  the  ana- 
lytical tools  they  use  to  make  internal  decisions 
on  how  to  allocate  their  resources  (295).  This 
chapter  presents  some  preliminary  evidence  re- 
garding the  current  and  potential  use  and  useful- 
ness of  cost-effective  analysis /cost-benefit  anal- 
ysis (CEA/CBA)  in  resource  allocation  decision- 
making in  HMOs.  It  does  not,  however,  exam- 
ine directly  the  question  of  the  cost  effectiveness 
of  HMOs  themselves. 

HMOs,  representing  the  prepaid  segment  of 
the  health  care  market,  are  both  insurors  and 
providers  of  health  care  (300).  Their  revenue 
comes  from  capitation  payments,  and  they  are 
responsible  for  delivering  care  to  an  enrolled 
population  (371).  Theoretically,  the  HMO  has  a 
direct  economic  incentive  to  provide  "cost- 
effective"  care.  Because  of  this,  some  people 
have  assumed  that  the  HMO  might  be  more  re- 
ceptive than  conventional  health  care  delivery 
settings  to  the  use  of  CEA/CBA  (615). 

It  is  important  to  note,  however,  that  HMOs 
exist  in  a  predominantly  fee-for-service  environ- 

'This  range  reflects  the  differences  observed  between  types  of 
HMOs.  Prepaid  group  practices  consistently  show  the  lowest  total 
(premium  plus  out-of-pocket  costs),  though  there  is  much  varia- 
tion among  them;  individual  practice  associations  show  among  the 
highest  costs '372). 


ment.  Since  potential  enrollees  have  a  choice  of 
providers,  and  physicians  a  choice  of  practice 
settings,  the  HMO  must  compete  for  both.  Be- 
cause its  financial  viability  depends  on  its  ability 
to  attract  and  retain  enrollees  (as  well  as  physi- 
cians), the  HMO  will  be  induced  to  offer  bene- 
fits and  services  comparable  to  those  offered  by 
its  fee-for-service  competitors. 

While  the  distinctive  characteristics  of  the 
HMO  guarantee  control  of  total  costs  (by  virtue 
of  capitation  payments)  and  promote  efficiency 
(by  virtue  of  the  HMO's  responsibility  to  deliver 
appropriate  services  to  an  enrolled  population 
within  a  constrained  budget),  there  is  nothing  to 
require  the  HMO  to  be  any  more  or  less  con- 
cerned with  the  effectiveness  and  benefits  of  a 
service  than  are  providers  in  other  health  care 
delivery  settings:  Rather,  the  benefits  side  of  the 
CEA/CBA  problem  is  largely  accepted  as  being 
defined  by  the  norms  of  "good  medical  practice" 
that  prevail  in  the  marketplace.  The  expecta- 
tions of  HMO  enrollees  and  physicians  tend  to 
diminish  the  importance  of  resource  allocation 
questions  pertaining  to  whether  or  not  a  benefit 
or  service  should  be  offered. 

In  general,  the  HMO  plan  bears  primary 
responsibility  for  the  financial  viability  of 
the  HMO  (in  a  management  or  administrative 
sense).  The  plan  faces  explicit  resource  alloca- 
tion decisions  upon  which  economic  incentives 
and  financial  constraints  directly  bear.  Whether 
or  not  physicians  are  sensitive  or  responsive  to 
these  incentives  and  constraints  in  making  their 
implicit  resource  allocation  decisions,  however, 
is  the  subject  of  considerable  debate  (152,371, 
589,590).  Some  claim  that  physicians  are  un- 
aware of  incentives  and  constraints,  and  even 
point  to  complete  coverage  as  giving  them  a  free 
hand  (372).  Yet  others  assert  that  physicians  in 
the  HMO  setting  are  sensitive  to  costs  (at  least 
on  a  subliminal  level)  and  that  even  a  decision, 
for  example,  to  treat  bronchitis,  or  to  perform  a 
coronary  bypass,  specifically  involves  the 
aspect  of  costs  (619). 
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In  fact,  the  available  evidence  shows  that 
almost  all  of  the  observed  cost  savings  in  HMOs 
are  attributable  to  lower  rates  (by  30  percent)  of 
hospitalization  for  HMO  enrollees  (373).  Deci- 
sions concerning  the  need  for  hospitalization  of 
HMO  enrollees  are  primarily  made  at  the  discre- 
tion of  attending  physicians.  It  is  true  that  the 
resource  allocation  decisions  made  at  the  ad- 
ministrative level  will  define  the  parameters  of 
decisions  faced  by  HMO  physicians.  Apart 
from  direct  and  indirect  controls  imposed  by 
the  plan  of  the  HMO,  however,  there  is  little 
consensus  on  any  one  explanation  of  why  HMO 
physicians  should  exhibit  this  apparent  "cost 
consciousness."  HMO  physicians,  trained  no 
differently  than  physicians  practicing  in  the  fee- 
for-service  environment,  bring  with  them  habits 
and  values  acquired  in  medical  school  or  in 
some  other  prior  practice  setting  (300).  Theo- 
retically, there  is  no  reason  to  expect  that  HMO 
physicians  would  be  any  more  likely  than  their 
fee-for-service  counterparts  to  explicitly  consid- 
er the  aspect  of  cost,  or  beyond  that,  to  consider 
cost  concomitantly  with  the  effectiveness  of  a 
service. 

If  anything,  the  HMO  physician  can  be  ex- 
pected to  allocate  the  available  resources  to  a 
given  medical  problem  foremostly  on  the  basis 
of  effectiveness  and  /or  efficacy,  differing  in  this 
respect  from  the  plan,  which  will  be  motivated 
to  allocate  resources  in  significant  part  on  the 
basis  of  costs,  i.e.,  efficiency.  No  further  at- 
tempt will  be  made  in  this  chapter  to  investigate 


the  use  and/or  usefulness  of  CEA/CBA  in  the 
decisionmaking  of  HMO  physicians. 

The  preliminary  evidence  and  conclusions 
that  are  presented  in  the  discussion  that  follows 
pertain  largely  to  the  resource  allocation  deci- 
sions made  at  the  organizational,  i.e.,  adminis- 
trative, level  of  the  HMO  plan.  The  evidence 
discussed  in  this  chapter  should  be  regarded  as 
tentative  because  of  the  great  diversity  in  types 
of  HMOs  and  the  small  number  of  HMOs  from 
which  the  evidence  was  gleaned.  The  influences 
assumed  to  determine  the  resource  allocation 
questions  relevant  to  the  HMO  theoretically 
derive  from  "generic"  characteristics  of  HMOs 
that  distinguish  them  from  other  health  care 
providers.  It  is  important  to  note,  however,  that 
there  is  probably  no  such  thing  as  a  "typical" 
HMO.  There  is  a  tendency  (in  the  rhetoric  and 
limited  literature  available  on  HMOs)  to  jux- 
tapose HMOs  against  "other  providers,"  but 
this  has  masked  important  differences  among 
HMOs  (295). 

The  findings  presented  here  do  seem  to  sub- 
stantiate the  preliminary  conclusion  that  formal 
CEA/CBA,  except  in  its  "net  cost  analysis" 
forms,  is  not  used  to  any  significant  degree  in 
decisionmaking  in  HMOs.  In  large  part,  the  evi- 
dence is  based  on:  1)  actual  resource  allocation 
decisions  recently  made  in  a  handful  of  HMOs 
around  the  country,  2)  the  analytic  techniques 
those  HMOs  used,  and  3)  the  role  that  analysis 
played  in  those  particular  decisions. 


LEGISLATIVE  HISTORY  AND  FEDERAL  SUPPORT 


HMOs  have  occupied  a  prominent  position  in 
Federal  health  policies  during  the  last  decade. 
They  have  been  promoted  as  one  strategy  for 
controlling  health  care  costs  and  encouraging  a 
more  rational  allocation  of  resources  to  health 
care  needs  (295). 2  Much  of  HMOs'  original  and 


2In  1979,  the  Office  of  Health  Maintenance  Organizations 
(OHMO)  contracted  for  a  CBA  to  examine  the  economic  costs  and 
benefits  of  continued  Federal  assistance  to  new  HMO  develop- 
ment. That  study  determined  the  estimated  rate  of  return  (in  cost- 
savings  to  the  community)  derived  from  the  Federal  HMO  devel- 
opment investments.  It  found  that  Federal  assistance  costs  are  re- 


sustaining  appeal  derives  from  the  fact  that 
these  organizations  have  been  viewed  as  a  more 
desirable  alternative  than  Federal  regulation  for 
achieving  those  objectives  (95). 

The  term  "health  maintenance  organization" 
was  the  brainchild  of  the  Federal  HMO  initia- 


covered  in  the  form  of  community  health  care  cost  savings  after  8 
years  of  HMO  operation,  and  projected  even  more  substantial  fu- 
ture savings  (302).  Based  on  these  findings,  OHMO  has  devised  a 
10-year  strategy  for  focusing  Federal  funds  and  support  of  HMO 
development  in  those  areas  that  offer  the  greatest  potential  return 
(489). 
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tive  in  the  early  1970's,3  but  the  concept  of  pre- 
paid group  practice  was  born  under  private 
sponsorship  and  developed  without  Federal  as- 
sistance before  1970  (372,  489).  In  1973,  when 
the  original  Health  Maintenance  Organizations 
Act  was  legislated  (Public  Law  93-222),  there 
were  39  prepaid  group  practices  in  existence, 
some  of  which  had  been  providing  care  since  the 
late  1920's  (489).  A  few  large  programs,  such  as 
Kaiser  Foundation  Medical  Care  Plan  (then,  as 
now,  the  Nation's  largest  HMO),  accounted  for 
the  bulk  of  enrollment. 

Based  on  the  performance  of  these  "proto- 
type" HMOs  already  in  operation,  advocacy  of 
the  HMO  approach  to  health  care  became  a  ma- 
jor Federal  initiative  in  1971  (589).  Federal  ef- 
forts culminated  in  the  enabling  legislation  for 
what  was  to  be  a  5-year  demonstration  pro- 
gram. This  legislation  (Public  Law  93-222)  pro- 
vided a  congressional  mandate  and  Federal  re- 
sources to  demonstrate  the  ".  .  .  extent  to  which 
the  HMO  concept  can  be  transplanted  to  new 
environments  with  new  participants  putting  it 
into  operation."  The  authority  has  been  ex- 
tended twice,  once  in  1976  (Public  Law  94-460) 
and  again  in  1978  (Public  Law  95-559)  (489). 

Federal  legislation  provides  explicit  directives 
on  how  an  HMO  should  be  organized  and  oper- 
ated. According  to  Federal  law,  HMOs  are  re- 
quired to  provide  both  ambulatory  and  hospital 
services.  The  law  recognizes  three  variations  in 
organization  that  distinguish  three  major  types 
of  HMOs  (discussed  below).  A  stringent  and 
precise  definition  of  the  federally  qualified 
HMO  mandates  a  very  comprehensive  benefits 
package  (including  such  benefits  as  mental 
health,  dental,  alcohol  treatment,  and  family 
planning  services),  sets  limits  on  charges  related 
to  utilization  (in  addition  to  premiums),  and 
specifies  enrollment  mechanisms  and  the  extent 


3Paul  Ellwood  is  credited  with  coining  the  term  "health  mainte- 
nance organizations"  in  1973  to  encompass  the  prepaid  group 
practices,  foundations  for  medical  care  (otherwise  know  as  indi- 
vidual practice  associations),  and  other  types  of  prepaid  systems  of 
health  care  delivery  (174). 


of  risk  sharing  (369,640).  Of  the  212  HMOs  in 
operation  as  of  April  1979,  only  99  (47  percent) 
qualified  under  this  strict  Federal  definition 
(489). 

Critics  of  the  legislation  have  viewed  it  as  un- 
fairly restrictive,  citing  a  lack  of  comparable  re- 
strictions in  the  health  insurance  industry  (372). 
Still,  increasing  levels  of  funding  have  been  au- 
thorized during  the  past  decade  for  the  con- 
tinued Federal  support  of  new  and  developing 
HMOs.  Approximately  97  (45  percent)  of  the 
212  HMOs  operational  in  April  1979  had  re- 
ceived some  sort  of  Federal  financial  assistance. 
Of  the  99  federally  qualified  HMOs,  74  (75  per- 
cent) have  received  Federal  assistance.  The 
Federal  HMO  program  is  responsible  for  a  great 
deal  of  growth  in  prepaid  plans  over  the  past 
decade  (489).  In  June  1979,  there  were  215 
HMOs  in  the  United  States  with  a  total  enroll- 
ment of  8,226,000  persons.  The  consistent  pro- 
motion of  the  HMO  program  by  Congress  re- 
flects the  opinion  that  Federal  support  of  HMOs 
can  yield  substantial  national  benefits. 

In  September  1979,  Congress  passed  amend- 
ments (Public  Law  96-620)  to  the  Health  Plan- 
ning Act  (Public  Law  93-641)  that  effectively  ex- 
empt HMOs  from  the  certificate-of-need  (CON) 
regulations  administered  by  local  health  systems 
agencies  (HSAs).  The  exemption,  based  on  evi- 
dence that  CON  regulations  were  being  used  to 
delay  or  prevent  HMO  development  (372),  re- 
flects congressional  support  of  the  broader  ra- 
tionale that  HMOs  may  generate  competition 
among  health  care  providers.  As  boldly  stated 
by  the  Office  of  Health  Maintenance  Organiza- 
tions (OHMO)  in  the  Department  of  Health  and 
Human  Services  (DHHS)  (489): 

.  .  .  HMOs  have  proven  to  be  a  quality,  cost- 
effective,  competitive  alternative  to  traditional 
fee-for-service  forms  of  medical  practice  .  .  . 
Both  Federal  support  and  substantial  private 
initiative  have  coalesced  over  the  past  several 
years  to  generate  a  new  momentum  for  future 
HMO  growth  ...  It  appears  that  HMOs  are 
moving  into  the  mainstream  of  the  health  deliv- 
ery system  of  this  country. 
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DEFINITIONS 

As  indicated  in  the  introduction,  there  is 
probably  no  "typical"  HMO.  That  notwith- 
standing, however,  all  HMOs  do  share  some 
characteristics  that  distinguish  them  from  other 
health  care  delivery  systems. 

A  "generic"  definition  identifies  a  minimum 
set  of  common  denominators  that  are  distinctive 
characteristics  of  HMOs  (372). 4  One  such  def- 
inition identifies  the  following  as  HMOs'  com- 
mon characteristic(s):  "the  provision  of  compre- 
hensive services  by  a  defined  set  of  physicians  to 
a  voluntarily  enrolled  population  paying  a  pro- 
spective per  capita  fee"  (95).  Luft  has  recently 
developed  and  elaborated  on  a  five-point  defini- 
tion of  the  "generic"  HMO  (372).  The  five  crite- 
ria he  identifies  are: 

1.  The  HMO  receives  a  fixed  annual  (or 
monthly)  premium  from  enrollees  that  is 
independent  of  their  actual  use  of  services. 

2.  The  HMO  assumes  at  least  part  of  the  fi- 
nancial risk  (or  gain). 

3.  The  HMO  serves  a  defined  population. 

4.  The  HMO  assumes  a  contractual  responsi- 
bility to  provide  services  to  its  enrollees. 

5.  HMO  enrollees  are  voluntarily  enrolled. 

Since  each  of  these  criteria  is  subject  to  being 
met  to  a  greater  or  lesser  degree,  Luft's  defini- 
tion allows  for  substantial  variation  among 
HMOs  that  meet  these  requirements  of  the  "ge- 
neric" HMO.  The  extent  to  which  these  criteria 
are  met  can  be  used  to  distinguish  between 
HMOs. 

Since  the  "generic"  definition  encompasses 
the  two  or  three  major  types  of  HMOs  generally 


"This  discussion  draws  heavily  from  a  draft  of  the  forthcoming 
book  by  Harold  Luft  entitled  Dimensions  of  HMO  Performance 
(372).  OTA  wishes  to  especially  acknowledge  Dr.  Luft  for  this 
helpful  conceptual  definition  of  HMOs,  of  which  there  are  (in  real- 
ity) virtually  limitless  varieties. 


recognized5 — the  prepaid  group  practice  (PGP), 
including  the  group  and  staff  models,  and  the 
individual  practice  association  (IPA)  model — 
additional  criteria  are  required  to  distinguish  be- 
tween these  types.6 

Two  additional  criteria  that  allow  the  distinc- 
tion between  PGP  and  IPA  types  to  be  made 
are:  1)  how  the  physicians  are  paid  by  the 
HMO;  and  2)  whether  they  provide  services  to 
enrollees  in  the  context  of  an  organized  group 
(closed  panel),  or  from  a  number  of  independent 
offices  (open  panel).  In  the  PGP  model,  physi- 
cians are  paid  on  either  a  salary  or  capitation 
basis.  In  contrast,  IPA  physicians  are  paid  by 
the  HMO  on  a  fee-for-service  basis  (according 
to  a  fee  schedule)  for  service  received  by  HMO 
enrollees.  Enrollees  in  the  IPA  model  HMO 
have  their  choice  of  physicians  from  those  be- 
longing to  the  IPA.  In  practice,  the  PGP  type  of 
HMO  clearly  predominates:  In  June  1979,  about 
63  percent  of  all  HMOs  were  PGPs,  but  PGPs 
served  about  84  percent  (or  6,942,000)  of  the 
over  8  million  total  HMO  enrollees  (300).  The 
distinction  between  group  and  staff  model  PGPs 
is  based  on  differences  in  the  nature  of  the  rela- 
tionship between  the  physicians  group  and  the 
plan.  In  the  group  model  PGP,  the  physicians 
are  a  distinct  legal  entity  that  contracts  with  the 
plan  of  the  HMO,  whereas  in  the  staff  model 
PGP,  the  physicians  are  hired  "on  the  staff"  of 
the  HMO. 


5A  fourth  kind  of  HMO,  the  network  model,  is  also  recognized 
by  some,  but  is  not  legally  recognized  (in  the  HMO  statute)  as  are 
the  prepaid  group  and  staff  models,  and  the  IPA  models  (640).  The 
Health  Insurance  Plan  of  Greater  New  York  is  the  oldest,  largest, 
and  best  known  of  the  network  models.  Though  network  models 
are  currently  fewest  in  number,  the  model  is  gaining  popularity, 
primarily  because  of  its  attractiveness  to  physicians  and  consumers 
alike,  while  maintaining  a  competitive  posture  in  the  community 
(413). 

These  characteristics  are:  1)  the  method  of  paying  the  key  deci- 
sionmakers, 2)  whether  physicians  are  full  or  part  time,  3)  the  ex- 
tent of  risk  sharing,  4)  whether  the  HMO  is  non-  or  for-profit,  5) 
whether  the  HMO  controls  its  hospital,  6)  whether  there  are  econ- 
omies of  scale,  and  7)  the  competitive  market  environment  faced 
by  the  HMO  (372). 
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INCENTIVES,  CONSTRAINTS,  AND  IMPERATIVES 


There  is  considerable  speculation  on  the  rela- 
tive influence  on  resource  allocation  questions 
relevant  to  the  HMO  of  the  internal  incentives 
and  budget  constraints  and  of  the  external  mar- 
ket environment.  These  two  influences  derive 
from  the  five  distinctive  characteristics  of  the 
"generic"  and  major  types  of  HMO.  The  inter- 
nal economic  incentives  and  constraints  arise 
from  the  combined  features  of  prepayment,  fi- 
nancial risk,  and  defined  population.  The  exter- 
nal competitive  imperative  stems  from  the  fea- 
ture of  voluntary  enrollment.  The  contractual 
responsibility  of  the  HMO  to  provide  services 
has  a  dual  effect,  acting  in  concert  with  both  sets 
of  characteristics  to  reinforce  their  respective  in- 
fluences on  resource  allocation  questions  rele- 
vant to  the  HMO. 

Economic  Incentives  and  Financial 
Constraints 

The  key  feature  of  prepayment  implies  two 
things  for  the  HMO:  1)  that  it  has  a  fixed  or  con- 
strained budget  (total  revenue  being  determined 
by  the  number  of  enrollees  and  the  fixed  premi- 
um rate),  and  2)  that  the  premium  paid  by 
HMO  enrollees  is  independent  of  their  actual 
use  of  services.  Thus,  the  usual  financial  incen- 
tives that  exist  in  fee-for-service  practice,  in 
which  the  net  income  of  the  provider  is  directly 
related  to  the  services  rendered,  are  reversed. 
Within  the  constraint  of  a  fixed  payment  per 
member  per  month,  an  HMO's  net  income,  to 
some  extent,  is  related  to  the  number  of  services 
provided.  Consequently,  there  is  a  direct  finan- 
cial incentive  to  provide  services  more  efficient- 
ly, or  to  provide  fewer  services  (presumably  re- 
ducing unnecessary  or  inappropriate  services), 
since  this  will  translate  into  lower  premiums  to 
enrollees  or  higher  profits  to  the  HMO,  or  both 
(615). 

The  economic  incentives  and  financial  con- 
straints deriving  from  prepayment  are  enforced 
by  the  second  criterion  which  says  that  the 
HMO  assumes  at  least  part  of  the  financial  risk 


(or  gain)  in  the  provision  of  services.  As  far  as 
the  "generic"  HMO  is  concerned,  this  feature 
only  implies  that  some  portion  of  the  burden  of 
risk  must  be  borne  by  the  plan  and  not  shifted  to 
a  third  party  through  reinsurance. 

Within  the  HMO,  however,  there  are  risk- 
sharing  arrangements  between  the  plan  and 
physicians.7  The  extent  of  "risk  sharing"  be- 
tween these  two  decisionmaking  entities  has 
been  tendered  as  the  theoretical  explanation  of 
observed  differences  between  the  performance 
of  HMOs  and  that  of  other  forms  of  health  care 
delivery.  But  the  extent  of  risk  borne  by  physi- 
cians varies  considerably  among  HMOs,  and 
generally  but  not  always,  by  type  of  HMO.  In 
the  IPA,  for  example,  the  plan  may  act  essen- 
tially as  a  third-party  insuror:  Reimbursed  by 
the  plan  on  a  fee-for-service  basis  (according  to 
a  fee  schedule)  for  services  rendered  to  enroll- 
ees, the  IPA  physician  (and  patient)  is  often 
somewhat  more  insulated  than  the  PGP  physi- 
cian from  the  financial  burden  associated  with 
decisions  regarding  the  services  actually  re- 
ceived. 

With  respect  to  hospitalization  rates,  both 
IPA  and  PGP  enrollees  had  considerably  lower 
rates  (25  and  35  percent  respectively)  than  con- 
ventionally insured  comparison  groups  (373). 
These  observed  differences  are  due  to  very  dif- 
ferent kinds  of  decisions  being  made  in  these 
HMOs.  In  the  case  of  the  IPA,  where  physicians 
are  often  at  lower  risk  for  the  implicit  resource 
allocation  decisions  that  they  make,  the  plan 
translates  its  own  economic  incentives  and  fi- 
nancial constraints  to  physicians  by  imposing 
implicit  controls  on  their  decisionmaking.  Deci- 
sions made  at  an  administrative  level  to  imple- 
ment such  mechanisms  as  preadmission  certifi- 
cation and  length-of-stay  review  translate  this 


7There  are  three  functional  parts  to  an  HMO,  which  may  even 
be  legally  distinct  entities:  1)  the  plan,  2)  physicians  (already  dis- 
cussed), and  3)  the  hospital  that  provides  inpatient  services  (372). 
Usually,  the  hospital  does  not  enter  into  risk-sharing  agreements 
with  the  HMO. 
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influence  and  assure  that  it  will  be  reflected  in 
decisions  made  at  the  provider  level.8 

Several  alternative  explanations  for  the  lower 
hospitalization  rates  of  PGP  enrollees,  and  for 
PGP  physician  behavior  in  general,  have  been 
tendered  and  debated  (152,590).  One  longstand- 
ing, simple  explanation  has  been  the  availability 
of  hospital  beds,  as  determined  by  explicit  re- 
source allocation  decisions  made  by  the  PGP 
plan  (152,590).  Such  constraints,  if  rigidly 
adhered  to,  could  be  effective  controls.  Evidence 
of  impact  on  physician  decisionmaking,  how- 
ever, is  minimal  (619).  Another  hypothesis,,  cur- 
rently supported  by  the  Group  Health  Associa- 
tion of  America  (GHAA)  and  others,  is  that  the 
combination  of  peer  pressures,  organizational 
factors,  and  capitation  motivates  physicians  to 
act  in  accord  with  professional  incentives  as 
well  as  the  constraints  of  the  HMO  (590).  A 
third  hypothesis  is  that  there  may  be  something 
"different,"  i.e.,  in  attitudes,  values,  etc.,  about 
physicians  who  choose  to  practice  in  an  HMO 
rather  than  in  a  more  traditional  fee-for-service 
practice  setting.  Although  this  explanation  is  in- 
tuitively appealing,  virtually  nothing  is  known 
about  the  possible  effects  of  "preselection"  on 
physician  behavior  in  the  HMO  setting  (372). 9 

The  influence  of  economic  incentives  and  fi- 
nancial constraints  on  resource  allocation  deci- 
sions faced  by  the  plan  and  physicians  in  the 
HMO  may  depend  on  the  risk-sharing  arrange- 
ments between  them.  This  characteristic  varies 
(generally)  by  type  of  HMO,  but  essentially  it  is 
the  plan  that  is  responsible  for  making  admin- 
istrative decisions  affecting  the  financial  viabil- 
ity of  the  organization,  while  physicians  are  still 
primarily  responsible  for  decisions  affecting  the 
well-being  of  HMO  enrollees.  The  influence  of 
physicians,  especially  heads  of  services  (e.g.,  a 
chief  of  surgery),  however,  is  often  substantial. 


"The  fee  schedule  used  to  set  reimbursement  rates  for  ambula- 
tory services  in  the  IPA  is  another  example  of  such  direct  controls 
imposed  on  physicians  by  the  plan.  Some  IPAs  have  imposed  very 
rigid  controls  on  physicians  (187). 

'There  is  also  a  self-selection  process  suspected  for  enrollees 
which  is  believed  to  affect  choice  and  disenrollment.  Clearly,  they 
see  the  plan  as  having  some  advantage,  as  witnessed  by  their  ten- 
dency to  display  "voice"  (rather  than  exit)  in  the  face  of  dissatisfac- 
tion with  services  (372). 


In  general,  theory  would  have  it  that  the 
greater  the  share  of  risk  borne  by  each  of  these 
entities,  the  greater  is  the  influence  of  these  in- 
centives and  constraints  on  their  decisions.  With 
respect  to  explaining  the  apparent  "cost-con- 
scious" behavior  of  HMO  physicians,  this  is  a 
tenable,  but  vigorously  contested,  hypothesis. 
With  respect  to  the  plan,  however,  the  criterion 
of  risk  assures  that  it  is  held  accountable  for 
meeting  the  conditions  fostered  by  prepayment: 
Its  impact  on  administrative  resource  allocation 
questions  made  by  the  plan  is  explicit  and  im- 
mediate. 

The  third  criterion  of  a  defined  population  ex- 
erts an  influence  only  insofar  as  it  determines 
the  total  revenue  available  to  the  HMO,  and  the 
nature  of  the  population  for  whom  it  must  pro- 
vide services.  The  defined  population  of  an 
HMO  currently  ranges  from  about  3,000  to  2.5 
million  enrollees  (the  latter  in  five  different  State 
regions)  (372).  Predominant  age  and  sex  charac- 
teristics of  the  enrolled  population  may  differ 
markedly  between  HMOs.  The  size  and  charac- 
ter of  the  HMO  population  will  directly  influ- 
ence the  types  of  decisions  to  be  made,  as  well  as 
the  administrative  structure  required  to  manage 
the  organization.  But  in  any  case,  at  any  given 
point  in  time,  the  HMO  can  project  its  annual 
revenue,  and  enrollees,  and  therefore  estimate 
the  demand  for  various  services.  Such  foresight 
facilitates  the  achievement  of  technical  efficien- 
cies, i.e.,  the  cost  per  unit  of  service,  and  also 
enables  the  HMO  to  make  tradeoffs  in  provid- 
ing benefits  and  services  that  will  be  appropriate 
to  its  particular  population. 

To  summarize,  the  internal  economic  incen- 
tives and  financial  constraints  faced  by  the 
HMO  derive  from  the  fact  that  the  HMO  plan 
has  finite  resources  to  allocate,  is  legally  respon- 
sible for  meeting  the  medical  needs  of  its  mem- 
ber population,  and  is  also  primarily  respon- 
sible for  maintaining  its  own  fiscal  viability. 
This  set  of  conditions,  arising  from  the  distinc- 
tive characteristics  of  prepayment  and  risk, 
guarantees  that  the  HMO  will  be  "cost  con- 
scious." But  the  internal  economic  incentives 
and  financial  constraints  do  not  necessarily  re- 
quire that  the  HMO  be  any  more  (or  less)  con- 
cerned with  efficacy,  or  effectiveness,  of  a  ben- 
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efit  or  service  in  allocating  its  resources  than 
any  other  health  care  delivery  setting  is. 

The  Competitive  Imperative 

This  influence  derives  chiefly  from  the  char- 
acteristic of  voluntary  enrollment,  which  im- 
plies that  prospective  enrollees  have  a  choice  be- 
tween alternative  health  care  providers  (and  in- 
surors),  and  that  they  have  the  option  to  disen- 
roll,  as  well  as  to  enroll,  in  the  HMO  (372).  The 
HMO,  existing  in  a  predominantly  fee-for-serv- 
ice  environment,  must  compete  for  its  enrollees: 
Its  financial  viability  depends  on  its  ability  to 
generate  revenue  through  enrollment,  and  con- 
sequently, on  its  ability  to  attract  enrollees. 

The  HMO  may  compete  on  the  basis  of  cost, 
as  well  as  the  range  and  quality  of  benefits  and 
services  it  can  offer.  In  this  regard,  HMOs  have 
a  competitive  "edge"  in  the  health  care  market, 
by  virtue  of  their  generally  much  lower  out-of- 
pocket  costs.  Competition  can  reintroduce  the 
concern  (by  consumers)  for  cost  (300).  In  the 
situation  where  HMOs  compete  against  one 
another,  the  limited  data  available  suggest  that 
relatively  small  differences  in  premiums  can  at- 
tract enrollees  (372).  But  when  premium  rates 
are  comparable,  the  question  becomes  one  of  of- 
fering the  most  (and/or  best)  services  for  the 
least  money  (95). 

The  ultimate  question  for  the  HMO  is  what 
impact  any  change  in  the  services  and  benefits 
offered  will  have  on  the  plan  premium.  Theoret- 
ically, the  use  of  "cost-effective"  techniques  in 
the  development  of  a  comparably  attractive,  yet 
low-cost,  product  would  seem  to  be  essential  to 
attract  and  retain  customers  in  a  competitive 
market.  At  the  least,  the  competitive  imperative 
does  not  negate  the  potential  for  viewing  re- 
source allocation  decisions  in  the  HMO  as 
CEA/CBA  problems. 

Ultimately,  however,  the  competitive  impera- 
tive means  that  the  HMO  must  meet  the  expec- 
tations of  enrollees  regarding  benefits  and  serv- 
ices, expectations  that  are  dictated  largely  by 
the  norms  of  practice  established  by  the  pre- 
dominating fee-for-service  mode  of  health  care 
delivery.  The  leverage  of  enrollees  to  demand 
these  services  stems  from  their  option  to  disen- 


roll.10  The  competitive  imperative  implies  that 
simply  because  competing  providers  are  offer- 
ing a  service,  and  doing  so  in  a  certain  manner, 
the  HMO  will  be  induced  to  provide  it  also,  and 
in  a  similar  fashion.  The  fact  that  HMOs  must 
also  compete  in  the  marketplace  for  physicians 
reinforces  this  tendency:  HMOs  will  seek  to 
provide  opportunities  for  medical  practice  com- 
parable to  those  offered  in  the  fee-for-service 
setting  in  order  to  attract  and  retain  physicians. 
This  situation  will  change  as  the  number  of 
HMOs  increases  and  they  are  forced  to  compete 
among  each  other  as  well  as  with  fee-for-service 
care. 

The  fact  that  HMOs  must  compete  for  both 
enrollees  and  physicians  means  that  the  HMO 
will  try  to  provide  benefits  and  services  similar 
to  those  available  from  other  insurors  and  pro- 
viders— but  within  its  financial  limits.  The 
HMO's  overall  strategy  in  making  resource  allo- 
cation decisions  will  be  to  maintain  or  improve 
its  competitive  position.  This  implies  that  any 
possible  efficiencies  achieved  in  providing  serv- 
ices, reductions  made  in  the  number  and/or 
range  of  services,  and/ or  tradeoffs  made  in  al- 
ternative ways  of  offering  services  that  might  be 
induced  by  the  internal  economic  incentives  and 
financial  constraints  will  be  subject  to  the  condi- 
tion that  the  HMO  remain  an  attractive,  com- 
petitive alternative. 

Thus,  given  both  a  competitive  imperative 
and  inherent  economic  incentives  and  financial 
constraints,  the  two  questions  most  commonly 
of  immediate  concern  to  the  HMO  are:  1)  how 
to  allocate  its  limited  resources  to  provide  the 
given  benefits  and  services  in  the  most  efficient 
manner  possible,  and/or  2)  how  to  reduce  inap- 
propriate services  and  benefits  (in  volume  and 
frequency)  without  a  perceptible  loss  in  quality 
(373). 

Evidence  appears  to  indicate  that  the  cost  sav- 
ings of  HMOs  are  due  to  changes  in  the  mix  and 
number  of  services  provided.  Overall,  ambula- 
tory services  received  by  HMO  enrollees  are  no 


10There  is  a  considerable  literature  regarding  what  is  essentially 
a  cost-benefit  decision  made  by  consumers  to  join  or  opt  out  of  an 
HMO:".  .  .  they  face  the  question  of  whether  the  benefits  they  are 
receiving  are  worth  the  premiums  they  are  paying"  (159). 
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different  from  those  received  by  persons  cov- 
ered under  conventional  insurance.11  (Among 
other  things,  this  reflects  the  imperative  of  the 
HMO  to  meet  the  consumer's  demand  for  ambu- 
latory services.)  Rather,  as  previously  indi- 
cated, almost  all  of  the  cost  savings  can  be 
directly  attributed  to  the  lower  (by  30  percent 
overall)  hospital  admission  rates  for  HMO  en- 
rollees  (373). 

Similarly,  empirical  data  suggest  that  persons 
conventionally  insured  with  full  coverage  for 
preventive  ambulatory  visits  receive  at  least  as 
many,  if  not  more,  preventive  services  than 
HMO  enrollees  (374).  In  contrast  to  the  ideolo- 
gy implied  in  the  name  "health  maintenance  or- 
ganization," the  apparent  greater  use  of  preven- 
tive services  in  HMOs  appears  to  be  attributable 
to  better  financial  coverage,  i.e.,  the  result  of 
lower  costs  of  ambulatory  visits  faced  by  enroll- 
ees (374). 

Because  of  its  financial  incentives  and  con- 
straints, the  HMO  (by  and  large)  tends  to  deliv- 
er a  "conservative"  brand  of  medical  care.  This 
implies  that  where  the  cost  of  a  preventive  serv- 
ice exceeds  that  of  treatment,  and  efficacy  is  un- 
clear, the  HMO  tends  not  to  provide  that  serv- 
ice, or  to  provide  less  of  it  (374).  With  respect  to 
preventive  services  that  are  of  questionable  ef- 
ficacy, it  will  tightly  ration  the  resources  re- 
quired to  provide  them.  For  example,  the  North- 
ern California  Kaiser  Foundation  Health  Plan 
shifted  its  stance  on  health  examinations  in  1976 
by  advocating  a  triannual  rather  than  an  annual 
physical  checkup  for  individuals  under  a  certain 
age. 

Thus,  in  allocating  resources  in  the  HMO,  the 
rationing  of  preventive  services  appears  to  be 
one  of  the  principal  questions  where  the  poten- 
tial benefits  (i.e.,  efficacy)  of  a  service  are  con- 
sidered in  relation  to  costs,  and  to  which  formal 
CEA/CBA  has  most  often  been  applied.  That 
HMOs  should  show  concern  for  the  efficacy  and 


"There  is  considerable  variation,  however,  in  ambulatory  visit 
rates  by  type  of  HMO.  IPA  model  HMOs  have  a  rate  almost  20 
percent  higher  than  that  of  conventionally  insured  persons;  PGP 
model  HMOs  have  slightly  varying  rates  that  even  out  to  be  only 
negligibly  different  from  the  rate  for  conventional  groups. 


effectiveness  of  preventive  (rather  than  surgical, 
for  instance)  services  in  relation  to  costs  may  be 
interpreted  as  another  reflection  of  their  adher- 
ence to  the  status  quo.  More  specifically  how- 
ever, it  is  the  low  "market  medical  benefit"  of 
preventive  services  that  allows  and  encourages 
questions  regarding  their  efficacy. 

In  summary,  the  limited  empirical  evidence 
available  regarding  HMOs  suggests  that  the  per- 
tinent resource  allocation  questions  and  deci- 
sions are  foremostly  influenced  by  the  competi- 
tive imperative,  but  are  also  ultimately  subject 
to  the  inherent  economic  incentives  and  finan- 
cial constraints  in  the  HMO. 

With  the  exception  of  preventive  (usually 
secondary)  services,  the  rationing  of  resources 
in  the  HMO  at  an  administrative  level  generally 
does  not  include  the  explicit  consideration  of  ef- 
ficacy relative  to  costs,  or  of  whether  or  not  a 
benefit  or  service  should  be  offered  on  that 
basis.  Instead,  the  competitive  imperative  in- 
troduces the  consideration  of  a  "market  medical 
benefit,"  or  the  qualitative  attribute(s)  of  a 
benefit  or  service  that  will  attract  and  retain 
physicians  and  enrollees  in  the  marketplace. 
While  these  are  not  societal  benefits,  they  are 
assumed  to  have  some  health  benefit  (619).  For 
HMOs,  these  benefits  are  largely  prescribed  by 
competitors  in  the  marketplace:  Within  HMOs, 
they  are  rationed  in  terms  of  how  and/or  how 
much  to  provide  (rather  than  whether  to  pro- 
vide them). 

By  and  large,  the  resource  allocation  ques- 
tions pertinent  to  this  task  are  concerned  with 
achieving  technical  efficiencies  and /or  volume 
and  frequency  processes.  These  are  questions 
generally  addressed  by  CEA/CBAs,  though  not 
by  those  that  take  health  effects  into  explicit  ac- 
count. The  concurrent  appreciation  and  consid- 
eration of  benefits  and  costs  in  HMOs  may  be 
regarded  as  implicit  "net  cost"  CEA/CBA,  and 
possibly  help  lead  to  the  "cost-effective"  be- 
havior of  the  organization.  The  remaining  dis- 
cussion in  this  chapter  supports  these  conclu- 
sions with  evidence  regarding  the  use  and  poten- 
tial usefulness  of  CEA/CBA  in  HMO  resource 
allocation  decisionmaking. 
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SURVEY  OF  HMOs 

Given  the  limited  scope  of  this  investigation, 
and  the  tremendous  diversity  possible  in  organi- 
zations characterized  as  HMOs,  the  HMOs  in- 
cluded in  this  small  survey  do  not  begin  to  ap- 
proximate a  representative  sample.  The  HMOs 
surveyed  here  were  identified  with  the  assist- 
ance of  several  individuals  who,  on  the  basis  of 
their  personal  knowledge  and  experience  with 
them,  suggested  various  organizations  that  they 
thought  might  be  most  likely  to  be  able  to  re- 
spond. 

The  final  selection  of  11  HMOs  was  based  on 
whether  each  HMO  had  recently  made  some  im- 
portant resource  allocation  decision,  since  this 
decision  was  to  provide  the  initial  basis  of  dis- 
cussion during  the  interview.  Table  5  indicates 
the  HMOs  interviewed,  specifying  for  each  the 
following  traits:  1)  type  of  HMO,  i.e.,  staff  or 
group  model  PGP,  or  IPA;  2)  age,  i.e.,  the 
length  of  time  in  operation;  and  3)  size,  i.e.,  the 
number  of  enrollees.  These  variables  were  rec- 
ognized as  potentially  important  influences  on 
findings.  In  the  small  survey  sample,  however, 
it  was  not  thought  worthwhile  to  try  to  select 
HMOs  according  to  these  traits. 


Only  one  or  two  individuals  were  interviewed 
at  each  HMO,  except  at  Kaiser,  where  sev- 
eral individuals  in  research  offices  were  inter- 
viewed.12 Consequently,  some  of  the  individuals 
interviewed  were  analysts,  some  were  adminis- 
trators, and  some  were  physicians.  But  some  in- 
dividuals acted  in  two  or  even  all  three  roles.13 

There  is  reason  to  be  skeptical  about  the  gen- 
eralizability  of  findings  regarding  the  usefulness 
of  CEA/CBA  in  these  few  HMOs.  Because  the 
HMOs  and  individuals  interviewed  were  se- 
lected on  the  basis  of  informed  (and  candid,  if 
not  objective)  opinions  as  to  their  ability  to  re- 


12Special  assistance  in  selecting  HMOs  and  the  individuals  in 
them  was  given  by  Mildred  Arrill  of  OHMO.  The  suggestions  of 
Frank  Seubold,  Harold  Luft,  Martha  Kendrick,  Jane  Willems,  and 
Morris  Collen  were  also  helpful  in  singling  out  the  several  orga- 
nizations and  persons  surveyed  here. 

"In  addition  to  the  individuals  interviewed  in  HMOs,  almost  as 
many  persons  from  OHMO,  trade  organizations,  and  scholars  of 
HMOs  were  interviewed.  Many  of  the  comments  were  useful  in 
both  anticipating  the  findings  and  drawing  conclusions  based  on 
them.  The  comments  of  these  individuals  are  referenced  through- 
out the  discussion.  Referenced  individuals  are  listed  in  the  bibliog- 
raphy. 


Table  5.— HMOs  Surveyed:  By  Type,  Age,  and  Size 


Name  of  HMO  and  location 

Type 

Age 

Size  (#  enrollees) 

1.  Capital  Area  (Albany)  Community  Health  Plan  

Staff  PGP 

3  years 

27,000 

Latham,  N.Y. 

2.  Arizona  Health  Plan  

Group  PGP 

8  years 

66,000 

Phoenix,  Ariz. 

3.  Group  Health  Cooperative  of  Puget  Sound  

Staff  PGP 

33  years 

270,000 

Seattle,  Wash. 

4.  Harvard  Community  Health  Plan  

Staff  PGP 

1 1  years 

85,000 

Boston,  Mass. 

5.  HMO  of  Pennsylvania  

IPA 

5  years 

42.000 

Willow  Grove,  Pa. 

6-8.  Kaiser  Foundation  Medical  Care  Plan3  

Group  PGP 

47  years 

Northern  California  Region,  San  Jose,  Calif  

1,637,050 

Southern  California  Region,  San  Diego,  Calif  

1,554,500 

Oregon  Region,  Portland,  Oreg  

231,743 

9.  Northern  Community  Health  Plan  

Group  PGP 

5  years 

23,000 

Glenview,  III. 

10.  Rhode  Island  Group  Health  Association  

Group  PGP 

8  years 

30,000 

Providence,  R.I. 

11.  Rutgers  Community  Health  Plan  

Staff  PGP 

4  years 

38,000 

New  Brunswick,  N.J. 

aln  addition  to  individuals  at  these  three  HMOs,  individuals  at  the  Kaiser  Permanente  Medical  Care  Program  research  facility  and  the  Kaiser  Foundation  Health  Services 
Research  Center  were  also  interviewed. 
SOURCE:  Office  of  Technology  Assessment. 
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spond  to  the  inquiry,  however,  these  findings 
may  represent  an  estimate  of  the  maximum  po- 
tential for  the  use  and  usefulness  of  CEA/CBA 


GENERAL  FINDINGS 

As  previously  mentioned,  the  discussions 
with  individuals  in  the  HMOs  centered  on  some 
specific  recent  resource  allocation  decision,  and 
whether  or  not  CEA/CBA  was  used,  or  would 
have  been  a  useful  analytical  technique  in  the 
decisionmaking  process.  The  resource  alloca- 
tion questions,  analytical  techniques  used,  and 
their  role  in  the  decisionmaking  process  were 
ascertained.  To  ascertain  the  type  of  analysis 
used,  a  brief  technical  description  was  obtained. 
Inquiry  regarding  the  usefulness  of  analysis  was 
concerned  with  determining  the  decisiveness  of 
its  impact  on  the  final  decision;  its  effect  of  a 
political,  ethical,  economic,  or  other  nature; 
and  its  contribution  to  the  decision  made  as 
viewed  by  the  various  parties  affected.  Each  of 
these  three  topics — resource  allocation  ques- 
tions, analysis  used,  and  the  usefulness  of  the 
analysis — are  discussed  below  with  respect  to 
both  the  current  and  potential  appropriateness 
of  CEA/CBA  in  HMO  decisionmaking.  In  addi- 
tion, the  extent  to  which  HMOs  have  institu- 
tionalized the  resource  allocation  process  is  dis- 
cussed. Finally,  the  current  and  potential  ap- 
plicability of  CEA/CBA  to  decisionmaking  in 
OHMO,  and  to  the  HMO  statutes  that  it  admin- 
isters, is  discussed. 

Current  Applicability  of  CEA/CBA 
in  HMOs 

Resource  Allocation  Questions 

A  wide  range  of  resource  allocation  questions 
were  reported  in  the  surveyed  HMOs.  Most  fre- 
quently, HMOs  were  engaged  in  a  simple  "make 
or  buy,"  "lease  or  purchase"  type  of  decision. 
The  question  was  not  so  much  whether  to  do 
something  at  all,  but  whether  something  was 
less  costly  to  do  "in"  or  "out"  of  the  HMO.  The 
most  commonly  cited  example  of  such  a  deci- 
sion was  made  with  respect  to  the  addition  of 
physician    staff,    usually    a    specialist  (i.e., 


in  HMOs.  Thus,  although  these  findings  should 
be  regarded  as  very  preliminary,  they  are  not 
without  significance. 


whether  to  contract  with  a  physician  in  the  com- 
munity, or  to  hire  another  physician  on  staff  of 
the  HMO).  Other  examples  of  "lease  or  buy" 
types  of  decisions  were  made  with  respect  to  ob- 
taining: 1)  improved  transcription  (of  medical 
records)  services;  2)  diagnostic  ultrasound 
equipment;  and  3)  new  physical  facilities.  The 
"make  or  buy"  decisions  arise  because  of  the 
capital  expenditure  issue  associated  with  them, 
but  more  importantly,  they  arise  from  the  im- 
perative exemplified  in  these  statements: 

Every  time  a  new  technology  comes  into 
medical  practice,  we  groan  because  we  have  to 
find  some  way  of  getting  it  .  .  .(527); 

If  the  "Docs"  come  to  us  and  say  they  want  to 
do  open  heart  surgery,  the  question  we  face  is 
not  whether,  but  how  we  can  do  it  (369);  and 

We  are  conservative,  practicing  "spartan" 
medicine,  (and)  perceived  of  as  "tightwads,"  but 
we  are  also  constrained  to  go  along  with  what  is 
practiced  in  the  medical  community  ...  If 
someone  introduces  something  new,  then  we 
have  no  choice  but  to  get  it  or  do  it  somehow 
(619). 

A  common  resource  allocation  question  in  the 
HMO,  therefore,  is  "How  can  we  provide  what 
is  demanded?" 

Other  resource  allocation  decisions  reported 
in  the  HMOs,  however,  tended  to  be  more  con- 
cerned with  whether  or  not  a  particular  service 
should  or  could  be  provided.  Some  of  these  de- 
cisions were  whether  to:  1)  implement  the 
COSTAR  (medical  information)  system  rather 
than  some  alternative  systems,  2)  install  a  cen- 
tralized v.  a  decentralized  automated  reception 
system,  3)  cease  doing  intestinal  bypass  surgery 
on  obese  patients,  4)  implement  a  health  educa- 
tion program  (2  cases),  and  5)  offer  biofeedback 
as  a  palliative  care  measure. 

Even  in  the  case  of  the  two-option  "make  or 
buy"  type  of  decision,  where  the  benefits  or  ef- 
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fects  are  generally  accepted  as  being  similar,  the 
concern  is  not  solely  for  efficiency.  "Relative  to 
just  saving  money,  there  is  (at  least)  the  assump- 
tion that  what's  being  done  is  of  benefit  .  .  .," 
either  to  the  organization  (in  the  case  of  admin- 
istrative decisions),  or  to  enrollees  (in  the  case  of 
medical  services);  the  benefit  is  assumed  (619). 
Often,  questions  about  benefits  were  identified 
on  the  basis  of  patient  or  physician  dissatisfac- 
tion with  current  practices  and  the  desire  for  im- 
proved quality  rather  than  a  less  expensive  way 
of  providing  the  service. 

In  the  case  of  questions  involving  the  aspect 
of  whether  to  provide  a  service,  however,  the 
relative  benefits  and  effects  of  various  alterna- 
tives were  explicitly  compared,  as  in  the  case  of 
COSTAR  and  the  automated  reception  system 
decisions.  In  the  case  of  one  of  the  health  educa- 
tion programs,  the  decision  involved  evaluating 
the  efficacy  associated  with  these  services.  In 
particular,  planners  were  specifically  asked  to 
measure  the  health  benefits,  i.e.,  how  is  this 
service  going  to  improve  health?  In  order  to  an- 
swer the  question  of  "should  we  spend  those 
dollars?"  the  health  benefits  (and  to  whom  they 
pertained)  had  to  be  proven.  In  the  case  of  bio- 
feedback, a  clinical  trial  of  that  technique  was 
being  undertaken  to  ascertain  whether  it  should 
be  made  available. 

In  all  questions,  the  associated  parameters 
were  limited  by  the  rigorous  constraints  of  time 
and  money.  Consequently,  much  of  the  re- 
source allocation  process  in  HMOs  could  be 
characterized  as  "problem-solving,"  in  which 
problems  were  often  identified  by  those  closest 
to  them.  For  example,  at  an  administrative  lev- 
el, there  is  a  great  deal  of  monitoring — of  hospi- 
tal admissions  and  length  of  stay,  referral  pat- 
terns among  physicians,  and  so  on.  In  the  case 
of  intestinal  bypass  surgery,  the  problem  was 
flagged  by  the  observation  that  "too  many" 
complications  were  developing  in  postsurgical 
obese  patients.  In  the  case  of  the  health  educa- 
tion program,  it  was  noted  that  the  HMO  was 
losing  its  "healthier"  enrollees,  because  "sicker" 
enrollees  impeded  their  access  to  HMO  services. 
In  that  case,  it  was  thought  that  a  health  educa- 
tion program  might  increase  enrollees'  satisfac- 
tion and  thus  help  retain  the  "healthier"  ones. 


Another  "problem  solving"  example  is  found  in 
the  implementation  of  a  health  appraisal  sys- 
tem. In  that  case,  the  problem  was  recognized, 
analyzed,  and  resolved  by  the  physician  in 
charge  within  the  medical  department.  The  phy- 
sician in  charge  faced  a  huge  backlog  of  initial 
physical  examinations  due  to  recent  rapid  ex- 
pansion of  the  HMO  and  limited  physician 
manpower.  On  the  basis  of  previous  research 
and  principles  of  multiphasic  health  testing 
(106),  he  devised  a  "linear  system"  for  "batch- 
ing" intake  examinations  and  freed  physician 
time  by  substituting  other  health  personnel  in- 
sofar as  possible. 

Thus,  much  of  the  resource  allocation  occur- 
ring in  HMOs  may  be  characterized  as  incre- 
mental decisionmaking  where  the  parameters 
are  subject  to  the  constraints  of  cost  and 
timeliness  of  a  solution  given  the  existing  situa- 
tion and  possibilities:  The  number  of  available 
options  is  usually  very  small.  Even  at  an  ad- 
ministrative level,  there  are  few  "strategic" 
resource  allocation  questions.  There  appears  to 
be  very  little  "zero-based"  decisionmaking  in 
HMOs.  The  HMO  rarely  faces  the  question  of 
whether  to  decentralize  or  centralize  (e.g.,  to 
have  two  or  six  physical  facilities).  Instead,  it  is 
faced  with  the  question  of  where  to  build  the 
next  facility. 

At  an  administrative  level,  HMOs  are  simply 
trying  to  be  more  efficient,  an  objective  which  is 
not  so  different  from  that  of  other  organization- 
al entities  delivering  health  care  (369).  "Cost  ef- 
fectiveness," however,  is  viewed  as  being  an  in- 
tegral consideration  in  the  decisionmaking  proc- 
ess at  all  times.  An  interviewee's  perception  of 
whether  the  resource  allocation  questions  ad- 
dressed by  HMOs  were  questions  of  cost  effec- 
tiveness depended  greatly  on  whether  the  in- 
dividual was  familiar  with  the  formal  technique 
of  CEA/ CBA  or  instead  had  a  more  "lay"  sense 
of  the  meaning  of  "cost  effectiveness"  (such  as 
that  exemplified  in  the  statement  "we  must  be 
cost  effective").  In  the  former  case,  the  resource 
allocation  questions  were  generally  not  viewed 
as  being  amenable  to  sophisticated,  health- 
effects-oriented  CEA/ CBA,  but  in  the  latter 
case,  they  were  viewed  as  problems  of  cost  ef- 
fectiveness. 
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Analyses  Used 

Generally  speaking,  formal  CEA/CBA  be- 
yond "net  cost"  analysis  was  not  found  to  be 
used  by  HMOs.  This  does  not  imply,  however, 
that  benefits  were  not  integrated  into  their  anal- 
yses. It  was  recognized  that  it  is  possible  to  be 
efficient  without  being  effective.  A  common 
perception  was  that  in  considering  every  major 
decision,  there  would  be  an  evaluation  as  to 
whether  the  service  would  be  "worthwhile."  The 
criteria  of  both  cost  and  quality  were  included 
in  making  this  determination:  The  first  question 
was  often,  "how  will  this  affect  the  quality  of 
service?"  and  the  second,  "what  is  the  impact  on 
the  cost  per  member  per  month?"  Analyses, 
therefore,  involved  the  evaluation  of  both  costs 
and  benefits:  Analyses  were  characterized  by 
the  objective  of  ascertaining  the  financial  impact 
of  a  given  (or  multiple)  option(s),  but  the  idea  of 
quality  loomed  over  all  considerations. 

Two  examples  illustrate  this  generalization  re- 
garding analyses  used  in  HMO  decisionmaking. 
The  example  of  whether  to  lease  or  purchase  ul- 
trasound equipment  illustrates  that  benefits 
were  also  taken  into  consideration  in  the  costing 
out  of  options,  and  that  the  question  of  medical 
efficacy  of  diagnostic  ultrasound  also  arose  in 
considering  those  benefits.  Questions  surround- 
ing this  "cost"  analysis  included  not  only  what 
the  costs  of  the  lease  option  would  be,  but  also 
whether  the  equipment  could  be  put  to  other 
uses  (besides  obstetrics)  in  the  HMO.  This  ques- 
tion involved  ascertaining  whether  ultrasound 
diagnostics  could  replace  other  diagnostic  tests, 
for  example,  CT  scans  in  some  cases,  and  the 
costing  out  of  such  "replacement"  effects.  Radi- 
ologists and  internists  in  the  HMO  were  ques- 
tioned on  the  potential  benefits  of  these  other 
uses  of  ultrasound.  These  physicians  concluded 
that  the  use  of  ultrasound  in  these  capacities 
would  still  have  to  be  backed  up  by  other  diag- 
nostic tests.  Consequently,  the  decision  was 
made  that  it  was  not  "worthwhile  enough"  to 
purchase  the  equipment  at  that  time,  and  the 
lease  option  was  chosen. 

The  second  example  of  the  level  of  detail  at- 
tained in  evaluating  the  costs  and  benefits  asso- 
ciated with  a  particular  decision  is  the  case  of 


the  COSTAR  medical  information  system.  Al- 
though, it  was  clear  from  the  outset  that  the 
COSTAR  system  would  probably  be  a  more  ex- 
pensive option,  it  was  also  recognized  that  it 
could  provide  a  completely  different  level  of 
service  in  the  HMO.  An  extremely  detailed 
analysis  of  per  dollar-impact  over  the  course  of 
several  years  was  conducted.  Part  of  this  anal- 
ysis involved  estimating  how  far  into  the  future 
the  various  options  would  work  and  included 
such  factors  as  the  rate  of  growth  (of  the  HMO) 
and  the  future  addition  of  a  (third  and  fourth) 
medical  center.  Qualitative  aspects  of  "after 
hours"  accessibility  and  the  continuity  of  care 
were  also  crucial  factors.  Even  potential  prob- 
lems of  future  energy  shortages  and  how  they 
could  affect  an  automated  recordkeeping  system 
were  considered.  Though  in  the  long  run,  a  pro- 
jected yearly  cost  difference  of  $20,000  to 
$40,000  more  for  COSTAR  v.  other  systems 
was  estimated,  COSTAR  was  chosen  on  the  ba- 
sis of  its  clear-cut  superiority,  i.e.,  its  "different" 
level  of  service. 

A  wide  range  of  analytical  expertise  and  re- 
sources was  found  to  be  applied  to  the  resource 
allocation  questions  in  diffferent  HMOs.  Prac- 
ticing physicians,  economists,  operations  re- 
searchers, and  accountants  in  various  adminis- 
trative and/ or  research  capacities  were  found  to 
be  "analysts:"  Even  in  small  HMOs,  where  the 
organization  is  run  very  "lean"  and  there  are 
few  spare  resources  to  devote  to  analyses  used 
in  decisionmaking,  analyses  were  being  con- 
ducted and  were  generally  more  sensitive  to  the 
costs  and  benefits  of  various  decisions  than 
might  have  been  expected.  Where  resources  al- 
lowed, as  in  the  case  of  a  health  education  pro- 
gram, methodological  expertise  was  sought  to 
deal  with  the  question  of  determining  the  asso- 
ciated health  benefits.  Only  the  very  largest 
HMO  was  found  to  have  conducted  a  formal 
CEA/CBA  "in-house"  (and  this  with  some  Fed- 
eral support).  By  and  large,  the  "in-house"  ef- 
forts and  resources  applied  in  pursuit  of  these 
questions  were  a  function  of  the  size,  age,  and 
financial  situation  of  the  particular  HMO. 
Though  HMOs  were  generally  thought  of  as  be- 
ing "too  small"  to  do  CEA/CBAs,  it  is  clear  that 
none  of  the  HMOs  in  this  limited  survey  were 
too  small  to  conduct  analyses  that  were  respon- 
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sive  to  the  perceived  information  needs  of  the 
problem  at  hand. 

The  analyses  themselves  ranged  from  "back- 
of-the-envelope"  to  rigorous  fiscal  analyses.  The 
consensus  opinion  was  that  most  resource  allo- 
cation questions  relevant  to  the  HMO  did  not 
require  formal,  sophisticated  CEA/CBA  to  as- 
certain a  satisfactory  answer  because  the  param- 
eters of  the  decisions  were  relatively  narrow.  In 
the  case  of  developing  and  implementing  a 
health  appraisal  system,  the  physician  in  charge 
did  a  "back-of-the-envelope  cost  analysis,"  in 
the  belief  that  if  cost  savings  (in  terms  of  substi- 
tion  of  physician  for  other  medical  manpower) 
were  not  evident  at  that  crude  level,  they  would 
probably  not  show  themselves  in  a  more  de- 
tailed version  of  analysis. 

It  was  commonly  appreciated  that  there  was 
never  enough  time  (or  money)  to  do  all  the  anal- 
ysis one  might  wish,  and  that  decisions  had  to 
be,  and  ultimately  are,  made  on  the  basis  of  the 
information  at  hand  at  the  time  the  decision  is 
made.  Concomitantly,  however,  there  was  an 
overall  confidence  in  HMO  staff's  comprehen- 
sion of  the  scope  of  most  resource  allocation 
questions  and  in  the  extent  and  depth  of  in- 
formation required  to  get  an  answer  that  was 
satisfactory,  if  not  absolutely  optimal. 

The  OTA  survey  examined  whether  the 
HMOs  borrowed  CEA/CBA  information  from 
external  sources  if  the  HMO's  own  resources 
were  not  sufficient  to  produce  such  information 
internally.  The  HMOs  reportedly  consulted  the 
medical  CEA/CBA  literature,  but  relatively  in- 
frequently used  it  as  a  basis  for  decisions.  The 
decisions  in  which  it  was  used  as  a  basis  (all 
made  by  a  single  HMO)  included  the  elimina- 
tion of  routine  chest  X-rays,  modification  of 
adult  physicial  examinations  and  pediatric  pre- 
vention schedules,  and  indications  for  CT  scans. 
In  other  cases,  including  electronic  fetal  moni- 
toring (EFM)  and  coronary  bypass  surgery  for 
angina,  the  literature  was  also  considered,  but 
was  regarded  as  too  equivocal  to  be  used  as  the 
basis  of  deciding  to  withhold  these  services. 

Thus,  while  being  "cost  effective"  was  viewed 
as  essential,  the  rigorous  forms  of  CEA/CBA 
were  seen  as  being  largely  irrelevant  in  these 


HMOs.  Rigorous  CEA/  CBA  studies,  among  the 
HMOs  surveyed,  were  found  to  be  very  rarely 
required  by  resource  allocation  questions,  even 
more  rarely  conducted,  and  only  infrequently 
borrowed  and  applied. 

Role  in  Decisionmaking 

A  general  consensus  on  the  role  of  analysis  in 
the  resource  allocation  decisions  found  in  this 
survey  is  summarized  by  the  following  state- 
ment: "The  analyses  are  critical,  .  .  .  (we) 
couldn't  make  the  decisions  without  them,  but 
the  analyses  do  not  make  the  decisions  .  .  ." 
(330).  Thus,  analyses  were  widely  appreciated 
as  an  integral  part  of  the  decisionmaking  proc- 
ess in  HMOs,  even  though  that  process  is  still 
perceived  as  an  essentially  political  one.  Impor- 
tant functions  of  these  analyses  were  the  follow- 
ing: 1)  to  define  the  parameters  of  the  decision 
at  hand,  2)  to  clarify  the  impacts  of  various  op- 
tions, and  3)  to  serve  as  a  basis  for  the  discus- 
sion of  issues  associated  with  a  particular  deci- 
sion. 

These  generalizations  are  exemplified  in  the 
case  of  COSTAR.  The  main  impact  of  that  anal- 
ysis was  said  to  be  that  it  was  ".  .  .  the  basis  for 
bringing  all  the  issues  forward  in  a  political 
power  struggle  .  .  ."(568). 

The  aforementioned  observations  suggest 
several  conclusions  regarding  the  usefulness  of 
analyses  in  the  decisionmaking  process.  First, 
they  indicate  that  when  introduced  into  the 
"political"  decisionmaking  environment,  any 
"objective"  analysis  will  come  under  harsh 
scrutiny  by  those  representing  the  opposing 
viewpoint.  They  also  suggest  the  constraints  of 
time  and  resources  within  which  analysts  and 
HMO  managers  and  other  decisionmakers  must 
operate,  i.e.,  how  few  options  present  them- 
selves, and  how  many  fewer  still  may  be  ana- 
lyzed and  considered.  They  also  suggest  that 
among  these  alternatives,  the  option  about 
which  analysts  and  decisionmakers  may  have 
the  most  information,  or  with  which  they  are 
most  familiar,  may  have  a  greater  likelihood  of 
being  chosen:  The  decision  made  will  often  be 
the  one  where  the  most  detailed  information  is 
available.  While  this  may  not  be  the  "best"  or 
optimal  choice,  it  is  a  rational  one. 
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Many  of  the  cost  analyses  might  technically 
be  termed  "net  cost"  analyses,  in  one  sense  an 
extreme  of  CEA/CBA.  Yet  when  these  analyses 
were  incorporated  into  the  decisionmaking 
process,  decisionmakers  became  extremely  sen- 
sitive to  the  aspects  of  quality  and  benefits: 
Though  it  was  not  considered  to  be  the  most 
"cost-effective"  option,  i.e.,  in  the  sense  of  least 
cost,  the  COSTAR  option  was  chosen  anyway 
on  the  basis  of  its  clearly  superior  capabilities 
(as  well  as  power  politics). 

The  COSTAR  example  illustrates  that  per- 
haps the  first  reason  to  chose  a  particular  option 
is  so-called  "program  effectiveness."  At  a  sub- 
liminal level,  it  is  physician  and  enrollee  satis- 
faction which  is  the  benefit  of  foremost  concern. 
There  is  great  hesitance  to  do  anything  that  does 
not  have  readily  discernible  benefit,  and  any- 
thing without  an  immediately  perceived  value 
(however  defined)  will  probably  not  be  done  at 
all. 

We  have  a  long  history  of  taking  care  of  the 
medical  needs  of  people.  It  is  very  difficult  to 
justify  withholding  something  on  the  basis  of 
analyses  in  general,  and  on  the  criterion  of  costs 
in  particular  .  .  .  What  people  want  is  what  is 
important  .  .  .  (527) 

Institutionalization  of 
Decisionmaking  Processes 

The  degree  to  which  resource  allocation  deci- 
sionmaking and  the  analyses  were  formalized, 
or  institutionalized,  is  (to  some  extent)  a  func- 
tion of  the  size  and  financial  situation  of  the 
HMO,  and  consequently,  of  the  administrative 
structure  required  to  manage  it.  The  majority  of 
examples  of  resource  allocation  decisions  dis- 
cussed so  far  may  be  characterized  as  "problem 
solving,"  where  those  individuals  closest  to  the 
problem  have  identified  and  possibly  even  ana- 
lyzed and  resolved  the  problem.  In  general, 
these  persons  were  not  "handed"  solutions  (i.e., 
resource  allocation  decisions)  by  managers  from 
higher  echelons  of  the  administrative  hierarchy. 
Such  "problem  solving"  was  critical  to  the  effi- 
cient allocation  of  resources  within  particular 
HMOs,  and  is  a  standard,  ongoing  feature,  even 
if  it  is  not  a  formalized  procedure. 


Institutionalized,  formal  analysis  for  resource 
allocation  decisionmaking  was  the  rare  excep- 
tion which  proved  this  rule.  The  decisionmak- 
ing process  meeting  these  criteria  that  was  most 
often  cited  was  the  "budgetary  process."  It  was 
often  perceived  that  CEA/CBA  takes  its  most 
explicit  form  in  the  HMO  in  the  annual  budget- 
setting  process.  In  HMOs,  both  large  and  small, 
the  budgetary  process  involved  projecting  the 
annual  revenue  (based  on  forecasts  of  member- 
ship), and  estimating  the  budget  (based  on  esti- 
mates of  demand  for  services  by  that  popula- 
tion). 

In  two  of  the  largest  and  oldest  HMOs,  Kaiser 
Foundation  Medical  Care  Program  (KFMCP) 
and  Group  Health  Cooperative  of  Puget  Sound 
(GHC),  examples  of  very  rigorous,  institution- 
alized procedures  for  allocating  resources  were 
found.  Without  delving  into  the  complex  orga- 
nizational and  administrative  structure  of 
Kaiser,  it  is  sufficient  to  say  that:  1)  there  is  an 
overall  vertical  hierarchy  within  both  the  over- 
all organization  and  in  each  of  the  five  regions, 
and  2)  that  the  program  has  established  consid- 
erable (relative  to  any  other  HMO)  research  ca- 
pabilities and  "in-house"  analytical  expertise.  In 
both  the  KFMCP  Central  Office  and  in  the 
Southern  California  Regional  Office,  there  are 
what  are  called  "benefit-cost  offices."  These  of- 
fices essentially  perform  financial  impact  anal- 
yses, i.e.,  they  examine  the  impact  of  a  pro- 
posed change  in  resource  allocation  on  the  cost 
per  member  per  month.  According  to  the  direc- 
tor of  one  of  them:  "...  We  do  not  do  cost- 
benefit  analysis  here — at  least  like  they  are  done 
in  academic  or  government  institutions  .  .  . 
where  the  bottom  line  is  life-years  saved,  or 
some  other  society-based  measure  .  .  ."  (527). 
Analysis  of  both  costs  and  benefits  is  based  on 
the  internal  economics  of  KFMCP.  Thus,  Kaiser 
does  appear  to  conduct  some  "program-perspec- 
tive" CEA/CB  As. 

At  least  in  the  Northern  California  Office,  re- 
quests for  analyses  are  approached  in  a  stand- 
ardized fashion,  with  preprinted  worksheets  be- 
ing used  to  evaluate  various  proposed  changes. 
Discounting  (usually  at  the  rate  of  10  percent) 
and  sensitivity  analysis  are  a  standard  part  of 
the  evaluation.  The  analyses  typically  examine 
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outcome  measures  such  as  total  net  benefit  and 
internal  rate  of  return  on  investment,  as  well  as 
a  cost-benefit  ratio.  They  are  key  planning  tools 
in  daily  use  at  Kaiser,  and  results  are  regarded 
as  proprietary  information.  Both  the  perform- 
ance and  use  of  these  analytical  tools  have  been 
incorporated  in  the  organizational  structure  and 
administrative  processes  of  KFMCP. 

Two  other  examples  of  the  institutionaliza- 
tion of  research  capabilities  and  analytical  ex- 
pertise in  Kaiser  are  found  in:  1)  the  Division  of 
Technology  Assessment  in  Oakland,  Calif.;14 
and  2)  the  Health  Services  Research  Center  of 
the  Bess  Kaiser  Foundation  in  Portland,  Oreg. 
At  the  former,  the  assessment  of  the  efficacy  of 
biofeedback  as  a  palliative  care  measure  for  var- 
ious conditions  is  an  example  of  the  research 
conducted  in  the  relatively  new  analytical  enti- 
ty. This  research  will  be  one  input,  along  with 
".  .  .  organizational  impact,  long-term  cost 
considerations,  and  patient  satisfaction,  as  well 
as  other  alternative  measures  .  .  ."  in  making 
the  decision  of  whether  or  not  to  include  bio- 
feedback as  a  benefit  (300). 

The  Health  Services  Research  Center  has  pub- 
lished several  studies  on  the  "cost  effectiveness" 
of  the  substitution  of  nurse  practitioners  and/or 
physicians  assistants  for  physician  manpower  in 
the  HMO  setting  (496,497).  Research  on  "do- 
not-admit"  surgery  was  also  conducted  at  this 
center  (263). 15 

The  results  of  both  have  been  used  through- 
out KFMCP  (as  well  as  other  HMOs)  resulting 
in  efficiencies  and  thus,  cost  savings.  Interest- 
ingly, although 

.  .  .  there  was  never  a  conscious  decision  made 
(by  management)  to  pursue  a  "do-not-admit" 
strategy,  there  were  soon  20  percent  of  all  sur- 
geries being  done  on  a  "do-not-admit"  basis.  As 
the  idea  "caught  on,"  and  administrative  sup- 
port developed,  40  percent  of  surgery  was  soon 
being  done  in  this  way  ....  The  cost  implica- 
tions of  this  change  (in  physician  behavior)  were 
enormous  .  .  .  (263) 


"This  new  Division  was  created  in  the  summer  of  1979,  and  is 
currently  headed  by  Dr.  Morris  F.  Collen. 

l5"Do-not-admit"  surgery  refers  to  cases  in  which  relatively  sim- 
ple surgical  procedures  are  performed  in  the  operating  room,  but 
the  patient  is  not  admitted  unless  there  are  complications  (263). 


This  and  the  nurse  practitioner/physician  assist- 
ant example  illustrate  the  use  of  formal  meth- 
odologies to  evaluate  various  configurations  of 
resources,  but  the  analyses  were  only  loosely, 
and  sometimes  not  at  all,  related  to  "across  the 
board"  policy  decisions  made  by  management. 

At  GHC  in  Seattle,  the  Pharmaceutical  and 
Therapeutics  Committee  is  a  specific  example  of 
the  institutionalization  of  cost-effectiveness  and 
cost-benefit  considerations  in  organizational  de- 
cisionmaking. Prescription  drugs,  included  in 
the  benefits  package  of  GHC  enrollees,  are  sub- 
ject to  review  and  approval  by  this  committee: 
The  objective  is  to  regulate  the  availability  of 
prescription  drugs  (within  the  plan).  No  GHC 
physician  may  introduce  a  pharmaceutical  into 
the  "stockpile"  of  those  already  available  unless 
it  has  been  passed  by  this  committee.  The  proc- 
ess is  similar  to  those  of  formularies  in  hospi- 
tals. For  example,  with  the  appearance  of  a 
"new"  antibiotic  on  the  market,  an  "across-the- 
board"  decision  will  be  made  to  use  either  the 
new  or  the  old  drug,  but  not  both,  in  that  HMO. 

In  considering  the  addition  of  another  drug, 
some  of  the  questions  asked  by  the  committee 
were  (given  in  this  order):  1)  Is  it  effective?  2)  Is 
it  costly?  3)  Are  there  other,  and  perhaps  too 
many  other,  options  already  in  stock?  and 
4)  What  is  its  effectiveness  and  cost  relative  to 
these  options?  Data  for  these  decisions  were 
taken  from  the  current  available  literature, 
clinical  experience,  and  so  on.  It  was  claimed 
that  this  control  resulted  in  very  substantial  (50 
percent)  reductions  in  the  cost  per  (comparable) 
prescription  for  GHC  enrollees.  The  committee 
is  an  example  of  a  formal  review  mechanism  in 
one  HMO  that  has  institutionalized  the  consid- 
eration of  "cost  effectiveness"  as  a  decision 
criteria — and  specifically  the  medical  (or  health) 
benefit — if  not  the  actual  conduct  of  formal 
analyses. 

Potential  Applicability  of 
CEA/CBAin  HMOs 

Resource  Allocation  Questions 

The  question  of  whether  something  is  cost  ef- 
fective or  cost  beneficial — in  the  most  formal 
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analytic  sense  of  the  term — is  by  and  large  an 
academic  one  in  HMOs.  As  perceived  by  the  Di- 
rector of  the  Kaiser  Health  Services  Research 
Center  (262): 

.  .  .  posing  the  question  inherent  in  these 
analyses  is  like  asking  ".  .  .if  the  money  supply 
is  tightened  in  our  economy,  then  .  .  ."  These 
analyses  are  too  broad  and  inclusive,  while  the 
problems  of  management  are  much  more  con- 
strained: CEA/CBA  do  not  fit  the  parameters  of 
management  decisions. 

Another  criticism  of  CEA/CBA  voiced  by 
both  analysts  and  decisionmakers  was  that, 
besides  insufficient  time  and  money,  there  was 
inadequate  freedom  to  ask  such  questions 
(619): 

We  must  serve  the  same  psychological  and  so- 
cial, as  well  as  medical,  needs  that  traditional 
providers  do  .  .  .  we  must  be  able  to  react  to  the 
same  emotional  and  social  pressures  .  .  .  (even 
though)  we  think  in  per  capita  terms  .  .  . 

There  are  two  reasons  why  ".  .  .  it  would  be 
very  difficult  to  justify  withholding  a  service  on 
the  basis  of  a  CEA/CBA  ..."  (527).  First,  as  in 
the  fee-for-service  practice  setting  ".  .  .an  ethi- 
cal/medical question  intervenes  in  trying  to  use 
CEA/CBA  as  a  basis  for  making  resource 
allocation  decisions  ..."  (588).  Secondly,  the 
HMO  faces  competitive  pressure  to  offer  as  sim- 
ilar (in  appearance)  services  as  possible  to  those 
of  fee-for-service  providers  in  order  to  meet  the 
expectations  of  both  physicians  and  enrollees  re- 
garding comprehensiveness  and  quality  of  serv- 
ice. 

Decisions  involving  ethical  and  moral  ques- 
tions, as  well  as  those  that  are  otherwise  value 
laden,  are  perceived  as  "risky"  areas  for  the  rig- 
orous application  of  analysis  such  as 
CEA/CBA.  For  example  (619): 

.  .  .  although  the  availability  of  hospital  beds 
may  be  held  at  a  very  finely  tuned  ratio  of  1.5  or 
1.6  beds  per  1,000  enrollees,  when  there  is  a 
"crunch,"  then  (we)  physicians  simply  seek  other 
available  hospial  beds  in  the  area  .  .  .  we  don't 
not  admit  patients  who  in  our  judgment  require 
hospitalization  .  .  . 

Similarly,  the  implementation  of  a  midwife  pro- 
gram initiated  by  enrollees  would  not  be  a  deci- 


sion likely  to  be  subjected  to,  or  influenced  by,  a 
formal  analysis. 

Decisionmakers  will  not  be  inclined  to  rely  on 
analysis  in  addressing  resource  allocation  ques- 
tions where  analysis  could  too  easily  (because  of 
uncertainties)  lead  to  the  "wrong"  answers. 

Generally  speaking,  the  question  of  whether 
something  is  cost  effective  or  cost  beneficial  in 
the  absolute  sense  is  seen  appropriately  as  hav- 
ing little  overall  practical  utility  in  the  HMO. 
This  view  was  expressed  by  two  high-ranking 
officials  of  GHAA: 

.  .  .  There  is  a  misplaced  emphasis  on  the  "ab- 
solute" cost-effectiveness  of  capital  intensive  in- 
vestment like  lab  analyzers,  or  CAT  scanners,  or 
open  heart  surgery  facilities  when  the  more  im- 
portant question  is  how  (at  what  level  or  intensi- 
ty) these  are  utilized  .  .  .  (588); 

and 

.  .  .  Much  of  the  cost-effectiveness  literature 
is  around  decisions  on  individual  technologies, 
but  it  is  the  decisions  regarding  volume  and  fre- 
quency processes  (such  as  laboratory  and  other 
diagnostic  testing)  where  there  is  the  greatest  po- 
tential for  cost  savings  .  .  .  (299). 

HMOs  seem  to  be  more  straightforwardly 
concerned  with  returns  to  scale,  returns  on  in- 
vestments, etc.  Thus,  in  HMOs,  "cost  effective- 
ness" is  correctly  interpreted  as  a  relative,  rather 
than  an  absolute,  attribute:  There  may  be  many 
"levels"  of  "cost  effectiveness"  associated  with  a 
particular  option  rather  than  an  answer  to  the 
binary  question  of  whether  or  not  an  option  is 
"cost  effective." 

Analyses  and  Role  in  Decisionmaking 

HMOs'  "in-house"  research  and  analytic  ca- 
pabilities were  usually  found  to  be  too  limited 
(in  terms  of  both  money  and  expertise)  to  allow 
the  conduct  of  rigorous  and  broad-based  CEA/ 
CBAs  or  clinical  trials  of  efficacy.  Other  poten- 
tial ways  of  obtaining  CEA/CBA  information 
are  through  technical  assistance  and  the  existing 
CEA/CBA  literature. 

Technical  assistance  by  which  CEA/CBA 
might  be  obtained  could  potentially  come  from 
three   sources:    1)   private   consulting  firms; 
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2)  GHAA,  the  trade  association  for  prepaid 
group  practice  type  HMOs;  and  3)  OHMO  in 
DHHS.  The  financial  resources  required  to  ob- 
tain this  from  private  consulting  firms  would 
make  this  source  an  infeasible  one  for  many 
(probably  most)  HMOs,  even  if  they  were  inter- 
ested in  pursuing  a  CEA/CBA  problem.  OHMO 
concedes  that  there  is  no  reason  why  CEA/CBA 
could  not  be  done  under  the  existing  statutory 
provision  for  general  technical  assistance  to 
HMOs  (159,313).  Although  it  is  conceivable 
that  a  CEA/CBA  problem  could  be  undertaken 
within  this  technical  assistance  provision,  the 
possibility  seems  remote  in  light  of  competing, 
and  more  pressing,  problems.  Finally,  GHAA 
also  brokers  and  provides  technical  assistance  to 
its  member  HMOs.  GHAA's  Associate  Director 
confidently  stated,  however  (588): 

...  I  would  expect  that  the  chances  of  an 
HMO's  approaching  us  to  do  a  CEA/CBA  are 
next  to  nothing  .  .  .  because  these  questions  just 
aren't  there  .  .  . 

Although  several  instances  were  found  where 
the  available  CEA/CBA  literature  was  con- 
sulted regarding  the  specific  technologies,  this 
resource  was  more  often  regarded  as  equivo- 
cal— both  because  of  the  technical  difficulties 
and  uncertainties  of  analyses,  and  the  lack  of 
available  efficacy  information  (619): 

.  .  .  (we)  have  looked  to  the  literature  for  a 
consensus  on  EFM,  ...  if  there  were  decisive 
evidence,  then  we  could  use  that  to  make  deci- 
sions regarding  its  use  .  .  .  but  there's  not 
.  .  .  (we)  wish  there  were,  but  there  isn't  .  .  . 

The  issues  of  transferability  of  existing  CEA/ 
CBAs  to  individual  HMOs  in  particular,  and  to 
HMOs  in  general,  adds  yet  an  additional  source 
of  uncertainty  (295): 

.  .  .  We  just  don't  have  the  health  delivery 
knowledge  (in  HMOs)  yet  to  be  able  to  predict 
"what  happens  if  'X'  happens?"  .  .  .  even  in  one 
HMO,  let  alone  to  generalize  .  .  . 

All  things  considered,  a  representative  consen- 
sus would  seem  to  be  that  "...  various  uncer- 
tainties make  formal  analyses  available  in  the 
literature  an  inadequate  basis  for  decisionmak- 
ing ..."  (615). 


Thus  it  seems  that  even  if  CEA/CBA  were  di- 
rectly relevant  to  the  resource  allocation  proc- 
esses in  HMOs,  most  HMOs  would  be  hard- 
pressed  either  to  produce  or  even  to  find  and 
borrow  CEA/CBA  information  that  would  be 
useful  to  their  individual  needs. 

The  receptivity  of  decisionmakers  to  analyses 
is  summarized  in  the  following  comments: 

.  .  .  when  a  CEA/CBA  will  tell  you  to  throw 
out  a  certain  option,  and  you  intuitively  feel  that 
it's  the  right  one  .  .  .  usually  you  disregard  the 
analysis  .  .  .  but  in  any  case  .  .  .  you  scrutinize 
and  challenge  the  analysis  .  .  .  (568); 

.  .  .  Decisionmakers  know  or  will  find  the 
limits  of  analytic  tools,  even  when  analysts 
don't,  because  if  the  analysis  is  to  be  used  as  a 
basis  for  decisionmaking  .  .  .  and  if  the  analysis 
turns  out  to  be  wrong,  it  is  the  manager  who 
must  endure  the  consequences  .  .  .  (295). 

Given  the  motivation — political,  ethical,  eco- 
nomic, or  other — to  do  so,  someone  can  cite 
enough  technical  problems  to  make  any  analysis 
equivocal.  Recommendations  for  changes  in  re- 
source allocation  based  on  analysis,  therefore, 
have  been  and  are  likely  to  continue  to  be  tenta- 
tive, with  pilot  and  demostration  projects  and 
considerable  nonanalytic  input  preferred  as  the 
basis  for  decisionmaking  (295). 

There  also  seem  to  be  misgivings  about  the 
"overall  post  facto"  nature  of  research  general- 
ly, and  about  analysis  that  tends  to  follow 
rather  than  precede  decisions.  As  noted  by  one 
observer  (588): 

.  .  .  CEA/CBA  are  in  "vogue"  again  .  .  . 
When  I  worked  for  CDC  (Center  for  Disease 
Control)  years  ago,  we  used  CBA  to  evaluate 
tuberculosis  vaccination  programs,  and  I  can  tell 
you  that  even  then,  the  answer  came  before  the 
analysis  was  ever  done,  and  it  always  justified 
the  decision  .  .  . 

Institutionalization  of 
Decisionmaking  Processes 

Although  it  was  found  that  the  conduct  of 
analyses  and  its  incorporation  in  HMO  deci- 
sionmaking was  typical  of  the  HMOs  surveyed, 
it  was  also  found  that  there  was  considerable 
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variation  among  HMOs  with  respect  to  whether 
and  how  these  processes  were  formalized.  An 
additional  finding  was  that  even  though  sophis- 
ticated forms  of  CEA/CBA  were  not  perceived 
as  relevant,  the  issues  perceived  to  underlie 
CEA/CBA  were  considered  an  essential  part  of 
the  decisionmaking  process  at  all  times. 

More  specifically,  however,  if  the  most  po- 
tentially pertinent  question  in  the  HMO  is  "at 
what  level  of  provision  is  'X'  cost-effective?" 
then  the  institutionalization  of  particular  orga- 
nizational procedures  such  as  peer  re'view  of 
patterns  of  use  of  various  resources,  what  con- 
stitutes inappropriate  use,  and  who  is  qualified 
to  designate  their  use  in  the  HMO,  as  well  as  the 
monitoring  of  referral  patterns,  hospital  admis- 
sions, and  the  like,  are  potentially  very  useful 
institutionalized  mechanisms  for  assuring  cost 
effectiveness  in  HMOs.  Many  HMOs  already 
have  instituted  some  of  these  formalities,  and  as 
a  result  have  achieved  dramatic  cost  savings 
(344). 

Because  of  the  great  diversity  of  HMOs  and 
variation  in  the  competitive  positions  they  hold 
in  their  respective  communities,  however,  it  is 
important  that  these  procedures  remain  "indi- 
vidualized" (rather  than  standardized),  i.e., 
each  according  to  the  HMO's  need  for  analyses 
and/or  review  (295).  Recently,  GHAA  has  rec- 
ommended to  the  council  of  the  National  Center 
for  Health  Care  Technology  that  a  study  be  un- 
dertaken with  the  purpose  of  developing  criteria 
for  the  acquisition  of  technological  resources  in 
HMOs.  The  proposed  study  would  indicate  the 
probable  impact  of  various  acquisitions  on  the 
costs  and  utilization  of  these  technologies  and 
their  relationship  to  per  capita  and  aggregate 
costs,  as  well  as  the  applicability  of  findings  to 
other  clinical  organizational  settings  (300).  The 
challenge  of  this  proposal  is  formidable  indeed. 
Still,  the  results  might  be  useful  if  posed  as  a 
relative  guide  or  as  one  model  to  follow. 

Statutory  Requirements  for 
CEA/CBA  in  HMOs 

Until  January  1,  1980,  the  regulations  pro- 
mulgated under  the  Health  Planning  and  Re- 
sources Development  Act  of  1974  (Public  Law 
93-641)  required  that  HMOs  demonstrate  the 


cost  effectiveness  of  their  requests  to  State 
health  planning  agencies  for  CON  approvals  of 
capital  expenditures.  During  1979,  an  extended 
process  of  developing  a  detailed  set  of  criteria  to 
be  used  to  ascertain  the  "cost  effectiveness"  of  a 
given  proposal  raged  on.  The  1979  amendments 
(Public  Law  96-79)  to  the  health  planning  law, 
however,  exempted  HMOs  from  CON  review 
by  local  HSAs  and  approval  by  State  planning 
agencies.  Neither  Federal  nor  non-Federal 
HMOs  are  now  covered  under  CON  regula- 
tions. Because  of  the  new  health  planning  law, 
the  controversial  endeavor  regarding  cost-effec- 
tiveness criteria  for  HMO  proposals  was  sus- 
pended. In  January  1980,  the  relationship  and 
responsibilities  of  HMOs  to  local  HSAs  and 
State  health  planning  agencies  were  described  as 
being  in  a  state  of  limbo  (159). 

OHMO  has  a  formalized  system  to  assist  po- 
tential or  young  HMOs  in  making  cost-effective 
resource  allocation  decisions.  HMOs  are  per- 
ceived as  businesses  by  OHMO,  and  the  main 
talent  and  skills  required  are  cited  as  those  of 
management.  Therefore,  under  the  authority  of 
the  technical  assistance  provisions  of  the  Health 
Maintenance  Organizations  Act,  OHMO  has 
developed  and  published  six  manuals  or  review 
guides  addressing  the  topics  of:  finance,  quality 
assurance  and  health  services  delivery,  manage- 
ment information  systems,  marketing,  structur- 
al and  contractual  relationships,  and  manage- 
ment arrangement  assessment  (308). 

To  aid  applicant  HMOs  in  making  cost-effec- 
tive decisions,  the  review  criteria  provide  cost 
ranges  and  reasonable  upper  cost  limits  at  vari- 
ous levels  of  enrollment  with  which  quoted  bids 
may  be  compared.  Similarly,  in  the  quality  as- 
surance and  health  services  delivery  review 
guide,  a  table  indicates  the  number  of  physi- 
cians required,  as  well  as  staffing  pattern  sug- 
gestions, at  various  levels  of  enrollment.  (These 
standardized  criteria  for  cost-effective  perform- 
ance are  intended  to  serve  as  benchmarks  only, 
and  are  not  meant  to  indicate  mandatory  stand- 
ards.) In  summary,  it  seems  that  the  analyses, 
decisionmaking  practices,  and  criteria  and 
standards  formally  institutionalized  in  the  ad- 
ministrative procedures  developed  by  OHMO 
are  very  similar  in  spirit  to  those  less  formalized 
in,  but  typical  of,  individual  HMOs. 
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Appendix  A.— Method  of  the  Study 


Most  of  the  studies  undertaken  at  OTA  rely  on  the 
advice  and  assistance  of  an  advisory  panel  of  experts. 
The  advisory  panel  for  a  particular  assessment  sug- 
gests source  materials,  subject  areas,  and  perspec- 
tives to  consider;  assists  in  interpreting  information 
and  points  of  view  that  are  assembled  by  OTA  staff; 
and  suggests  possible  conclusions  based  on  the  accu- 
mulation of  information  produced  by  the  study.  The 
panel  members  review  staff  and  contract  materials 
for  accuracy  and  validity,  discuss  policy  options  of 
the  study,  and  present  arguments  for  and  against  the 
options  and  conclusions.  They  do  not  determine  the 
report's  final  form,  however,  and  are  not  responsible 
for  its  content,  direction,  or  conclusions. 

The  advisory  panel  for  the  current  assessment  con- 
sisted of  12  experts  with  backgrounds  in  business, 
ethics,  health  policy,  law,  economics,  statistics,  and 
medicine.  The  panel  was  chaired  by  Dr.  John  R.  Hog- 
ness,  former  President  of  the  University  of  Washing- 
ton and  current  President  of  the  Association  of  Aca- 
demic Health  Centers.  Two  members  of  the  OTA 
Health  Program  Advisory  Committee,  Dr.  Stuart  H. 
Altman  and  Dr.  Frederick  Mosteller,  also  served  on 
the  advisory  panel. 

The  first  panel  meeting  was  held  in  Washington, 
D.C.  (the  site  of  all  four  panel  meetings),  on  Decem- 
ber 13,  1978.  Panel  members  reviewed  the  study  plan 
of  the  assessment,  examined  the  need  for  specific  case 
studies,  and  reviewed  a  number  of  suggestions  for 
potential  case  study  examinations.  The  case  study 
approach,  as  part  of  the  general  study  framework,  is 
intended  to  illustrate  the  uses  of  CEA  or  CBA  in 
health  care,  especially  to  evaluate  medical  technol- 
ogy, along  with  the  impacts,  and  strengths,  and 
weaknesses  of  these  uses.  The  advisory  panel  was  in- 
strumental in  helping  the  staff  set  goals  for  the  assess- 
ment, establish  boundaries  for  its  focus,  and  define 
the  basis  for  and  role  of  CEA  or  CBA  as  a  decision- 
assisting  tool  in  the  health  care  system.  The  panel 
was  also  helpful  in  identifying  public  and  private  sec- 
tor uses  and  users  of  CEA/CBA  techniques  or  infor- 
mation in  decisionmaking. 

To  help  select  medical  technologies  for  the  final  list 
of  case  studies,  the  following  criteria  were  devel- 
oped: 

•  examples  of  types  of  technology  by  function 
(preventive,  diagnostic,  therapeutic,  and  reha- 
bilitative); 

•  examples  of  different  stages  of  development  and 
diffusion  (not  yet  diffused,  experimental  or  pi- 
lot, established  in  medical  care); 


•  examples  from  different  areas  of  medicine  (such 
as  general  medical  practice,  pediatrics,  obstet- 
rics, radiology,  and  surgery); 

•  examples  addressing  medical  problems  that  are 
important  because  of  their  high  frequency  or  sig- 
nificant impacts; 

•  examples  of  technologies  with  associated  high 
costs; 

•  examples  of  technologies  in  widespread  use;  and 

•  examples  with  sufficient  evaluable  literature. 

On  the  basis  of  these  criteria  and  panel  recommen- 
dations, OTA  staff  selected  the  case  study  topics.  In 
addition,  the  Senate  Finance  Committee  had  specifi- 
cally requested  four  case  studies:  psychotherapy,  res- 
piratory therapy,  diagnostic  X-ray,  and  length-of- 
stay  in  hospitals.  Nineteen  case  studies  (identified  in 
app.  F)  were  added  to  the  study  plan  (Background 
Papers  #2,  #3,  and  #5). 

The  second  advisory  panel  meeting  was  held  on 
February  14,  1979.  The  panel  reviewed  the  literature 
gathered  by  OTA  staff,  suggested  additional  refer- 
ences and  sources  of  information,  and  evaluated  the 
case  study  plan.  In  addition,  it  examined  the  staff's 
preliminary  work  on  CEA  and  CBA  methodology  to 
identify  strengths,  weaknesses,  and  omissions.  The 
panel  was  also  helpful  in  commenting  on  the  decision 
area  papers  (e.g.,  use  of  CEA  in  reimbursement 
coverage  decisions)  that  were  being  prepared  by  the 
staff.  Finally,  it  was  asked  to  review  a  separate  study 
being  conducted  as  part  of  the  overall  assessment: 
The  Management  of  Health  Care  Technology  in  Ten 
Countries  (Background  Paper  #4). 

Several  subprojects  were  pursued:  a  survey  of  rele- 
vant State,  Federal,  local,  and  private  and  nonprofit 
agencies  and  organizations  to  determine  the  extent  of 
use  or  support  of  CEA/CBA  activities  (app.  B);  and 
an  extensive  review  of  the  health  care  literature  to 
develop  a  comprehensive  bibliography  of  health  care 
CEAs,  CBAs,  and  related  publications. 

The  psychotherapy  case  study  (Background  Paper 
#3)  used  a  separate,  additional  advisory  panel  that 
performed  functions  similar  to  the  ones  the  overall 
advisory  panel  performed  for  the  full  assessment. 
The  psychotherapy  panel  met  on  August  7,  1979,  in 
Washington,  D.C,  to  review  the  study  plan,  suggest 
improvements,  and  evaluate  the  background  work 
performed.  This  panel  met  again  on  November  15, 
1979,  to  review  and  critique  the  initial  draft  of  the 
case  study. 

The  third  meeting  of  the  advisory  panel  for  the  full 
assessment  took  place  on  October  16,  1979.  In  gener- 
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al,  the  panel  focused  on  reviewing  the  status  of  the 
various  parts  of  the  study.  It  was  also  asked  to  dis- 
cuss the  early  drafts  of  the  methodology  report 
(Background  Paper  #1)  and  of  the  use  of  CEA  in 
medicare  reimbursement  coverage. 

During  1979,  two  additional  subprojects  were  initi- 
ated: a  survey  of  analysts  who  had  performed  CEAs 
and  CBAs  of  health  care  technologies  (app.  C),  and  a 
paper  examining  the  ethical  implications  of  CEA  and 
CBA  techniques  (app.  D).  The  survey  of  analysts  fo- 
cused on  the  resource  "costs"  used  to  perform  actual 
CEAs  or  CBAs.  The  analysts  were  also  asked  to  esti- 
mate the  expected  or  potential  costs  of  an  "ideal" 
study  team  doing  CEA/CBA  analyses  of  health  care 
technology  on  a  routine  and  continuing  basis.  The 
second  subproject  was  a  paper  by  the  Hastings  Insti- 
tute on  the  ethical  considerations  of  conducting  and 
using  efficiency-based  analyses  such  as  CEA  and 
CBA  in  the  health  care  system. 

On  January  18,  1980,  the  authors  of  the  case  stud- 
ies assembled  in  Washington,  D.C.,  to  review  the  ap- 
plicability of  CEA/CBA  to  health  care  decisionmak- 
ing and  to  discuss  the  methodological  or  data  prob- 
lems they  faced  in  trying  to  apply  CEA/CBA  to  their 
case  study  areas.  Case  study  authors  also  discussed 
the  policy  issues  involved  in  the  use  of  CEA/CBA  in 
health  care  decisionmaking. 

The  final  meeting  of  the  advisory  panel  was  held 
on  March  28,  1980.  At  this  meeting,  the  panel  re- 


viewed drafts  of  the  summary  report,  including  the 
policy  options  for  congressional  consideration.  Using 
the  comments  generated  at  this  meeting,  OTA  staff 
revised  the  assessment  report  drafts. 

The  results  of  this  assessment  are  being  issued  in 
six  volumes  (described  in  app.  F).  A  two-  or  three- 
tiered  review  process  was  used  for  each  of  the  vol- 
umes and  for  each  individual  case  study.  The  initial 
drafts  were  reviewed  first  by  OTA  staff  and  advisory 
panel  members.  In  certain  instances,  outside  review- 
ers were  also  asked  for  comments.  After  the  authors 
completed  their  revisions  based  on  the  reviewers' 
suggestions  and  comments,  the  drafts  were  sent  out 
for  a  second  round  of  review  by  a  much  broader 
range  of  experts  in  a  diversity  of  settings:  Federal 
agencies,  State  or  local  offices,  private  and  nonprofit 
organizations,  academic  institutions,  practicing 
health  professionals,  consumer  groups,  and  other  se- 
lected individuals.  Altogether,  more  than  400  indi- 
viduals or  organizations  were  asked  to  comment  on 
drafts  of  case  studies  and  other  volumes  of  this  as- 
sessment in  the  second  round  of  review.  The  final  re- 
port, the  volume  containing  congressional  options, 
was  reviewed  by  more  than  100.  After  appropriate 
revisions  based  on  the  comments  received  had  been 
made,  drafts  were  prepared  for  a  final  review  by  the 
assessment's  advisory  panel,  by  the  Health  Program 
Advisory  Committee,  and  by  other  individual  re- 
viewers. 
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Introduction 

This  appendix  reports  on  the  results  of  an  OTA 
survey  of  the  use  of  cost-effectiveness  and  cost-bene- 
fit studies  by  the  major  Federal  health  agencies,  State 
and  local  governments,  and  nongovernmental  orga- 
nizations. For  the  most  part,  OTA's  survey  showed 
that  CEA/CBA  analyses  are  not  frequently  con- 
ducted or  applied  to  health  care  decisionmaking.  Fur- 
ther, when  such  analyses  are  used,  they  tend  to  be 
cost-benefit  rather  than  cost-effectiveness  analyses. 

Office  of  Assistant  Secretary  for 
Planning  and  Evaluation,  DHHS 

In  May  1966,  John  Gardner,  Secretary  of  HEW 
(now  DHHS),  established  five  program  analysis 
groups  to  conduct  CBAs  of  several  disease  control 
programs.  The  objective  of  this  effort  was  to  provide 
a  basis  for  comparing  alternate  programs  and  setting 
priorities  for  additional  funding.  Thus,  an  HEW  offi- 
cial responsible  for  overseeing  the  analyses  observed 
(266): 

HEW  supports,  or  could  support  a  number  of  cate- 
gorical disease  control  programs  whose  objectives  are, 
or  would  be,  to  save  lives  or  to  prevent  disability  by 
controlling  specific  diseases.  The  studies  were  there- 
fore an  attempt  to  answer  the  question:  If  additional 
money  were  to  be  allocated  to  disease  control  pro- 
grams, which  programs  would  show  the  highest  pay- 
off in  terms  of  lives  saved  and  disability  prevented  per 
dollar  spent? 

The  effort  was  originally  undertaken  in  response  to 
a  request  by  the  Bureau  of  the  Budget  for  thorough 
analysis  of  the  costs,  benefits,  and  objectives  of  ex- 
isting and  projected  programs.  That  request  reflected 
an  interest  in  attempting  to  rationalize  Government 
allocation  decisions  by  building  on  the  planning-pro- 
graming-budgeting  system  adopted  by  the  Defense 
Department  under  Secretary  McNamara  (637). 

Ultimately,  the  five  HEW  program  analysis  groups 
produced  several  cost-benefit  studies: 

1.  Disease  Control  Programs:  Arthritis,  1966. 

2.  Disease  Control  Programs:  Cancer,  1966. 

3.  Disease  Control  Programs:  Delivery  of  Health 
Services  for  the  Poor,  1967. 

4.  Disease  Control  Programs:  Kidney  Disease: 
Program  Analysis,  1967. 

5.  Program  Analysis:  Maternal  and  Child  Health 
Care  Programs,  1966. 

6.  Application  of  Benefit-Cost  Analysis  to  Motor 
Vehicle  Accidents,  1966. 


7.  Human  Investment  Programs:  Vocational  Re- 
habilitation, 1967. 1 
In  general,  the  impact  of  these  studies  appears  to 
have  been  very  limited  by  existing  political  and 
bureaucratic  considerations,  methodological  short- 
comings in  the  analyses,  and  unrealistic  expectations 
of  the  impact  such  evaluation  studies  could  have  on 
decisionmaking  (637). 

One  of  the  analyses,  however,  did  have  a  major 
impact.  The  analysis  of  maternal  and  child  health 
programs  that  examined  the  cost  savings  produced 
by  federally  sponsored  periodic  screening  for  low- 
income  children  played  a  major  role  in  congressional 
passage  of  the  Early  and  Periodic  Screening,  Diagno- 
sis, and  Treatment  (EPSDT)  Program  (H.R.  57100, 
1967)  (637). 2  The  Johnson  administration's  interest  in 
enacting  a  child  health  program  created  a  receptive 
audience  for  the  ideas  presented  in  the  analysis.  Diffi- 
culties in  the  implementation  of  EPSDT  have  arisen 
over  the  years.  One  of  the  major  obstacles  has  prov- 
en to  be  the  high  costs  that  States  incur  when  they 
participate  in  the  program  (213).  This  problem  was 
not  focused  on  in  the  original  examination  of  societal 
costs  and  benefits. 

National  Institutes  of  Health 

NIH  is  the  Government's  principal  biomedical  re- 
search agency.  Its  18  major  organizational  compo- 
nents support  research  on  the  causes,  diagnoses,  and 
treatment  of  diseases.  Although  a  few  institutes  have 
engaged  in  economic  analyses  and  cost-benefit 
studies,  such  efforts  have  been  relatively  atypical. 
The  institutes  have  traditionally  focused  almost  ex- 
clusively on  gathering  new  knowledge  as  it  relates  to 
the  disease  process,  having  added  an  emphasis  on 
assessing  the  safety  and  efficacy  of  new  and  existing 
technologies  only  recently.  In  addition,  NIH  has  a 
very  limited  capacity  to  conduct  economic  analyses. 

These  factors,  among  others,  have  combined  to 
discourage  NIH  from  conducting  CEA  or  CBA  stud- 
ies, as  noted  in  an  NIH  staff  memorandum  (481): 


■For  a  description  of  this  effort,  see  R.  N.  Grosse  and  W.  Smith,  "Cost-Ef- 
fectiveness and  Cost-Benefit  Analysis  for  Public  Health  Programs."  Public 
Health  Reports.  1968  (268). 

2For  description  of  this  analysis  and  the  role  played  in  the  passage  of 
EPSDT,  see  J.  Wholey  and  G.  Silver,  Maternal  and  Child  Health  Care  Pro- 
gram, 1966  (638);  A.  Levin,  "Cost  Effectiveness  in  Maternal  and  Child 
Health:  Implications  for  Program  Planning  and  Evaluation,"  N.  Eng.  ]. 
Med..  1968  (356);  A.  M.  Foltz,  "The  Development  of  Ambiguous  Federal 
Policy:  Early  and  Periodic  Screening,  Diagnosis  and  Treatment  (EPSDT)" 
(213),  Milbank  Memorial  Fund  Quarterly.  1975;  and  R.  N.  Grosse,  "Cost 
Benefit  Analysis  of  Health  Services,"  1972  (266). 
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The  Institutes  of  the  NIH  lack  the  staff,  funds,  and 
in-depth  expertise  to  undertake  large-scale  economic 
analyses.  Furthermore,  safety  and  efficacy  are  still  the 
primary  concerns  in  the  areas  of  disease  research  in 
which  NIH  is  involved,  the  state-of-the-art  in  many  of 
these  areas  is  not  yet  sufficiently  advanced  to  make 
cost-benefit  analyses  feasible.  However,  the  NIH 
shares  with  agencies  responsible  for  health  care  deliv- 
ery and  regulation  the  mandate  to  evaluate  new  medi- 
cal procedures  and  devices,  and  where  appropriate  the 
Institutes  do  conduct  small-scale  cost-benefit/cost-ef- 
fectiveness analyses  as  components  of  more  compre- 
hensive technology  assessment  studies.  Extensive  ef- 
forts at  cost-benefit  analyses  would  be  more  appropri- 
ately undertaken  by  other  federal  agencies  within 
DHEW  .  .  .  which  relate  more  closely  with  the  health 
care  [delivery]  sector. 

Economic  and  cost-benefit  studies  that  have  been 
conducted  by  NIH  have  generally  resulted  from  the 
interest  of  individual  staff  people  (480).  By  far  the 
most  common  types  of  economic  analyses  produced 
by  NIH  have  been  various  cost-of-illness  studies. 
These  have  been  used  in  both  planning  and  budget 
justifications.  NIH  cost-of-illness  studies  include: 

1.  National  Institutes  of  Health,  Office  of  the  Di- 
rector, "Trends  of  Economic  Costs  of  Illness," 
1979. 

2.  National  Cancer  Institute,  "Measurement  of 
the  Cost  of  Cancer  Care,"  1978. 

3.  P.  A.  Hoffstein,  K.  K.  Krueger,  and  R.  J.  Wine- 
man,  "Dialysis  Costs:  Results  of  a  Diverse 
Sample  Study,"  Kidney  International,  9:286, 
1976.  [This  is  a  summary  of  a  study  conducted 
for  the  National  Institute  of  Arthritis,  Metabo- 
lism, and  Digestive  Diseases  by  the  Nephrology 
Cost  Group.] 

4.  National  Institute  of  Arthritis,  Metabolism, 
and  Digestive  Diseases,  "The  Cost  and  Preva- 
lence of  Complications  of  Dialysis  in  Five  Di- 
alysis Centers,"  1978. 

5.  E.  J.  Sondik,  "Heart  Disease,  Magnitude  of  the 
Problem,"  report  prepared  for  the  National 
Heart,  Lung,  and  Blood  Institute  Task  Force  on 
Heart  Disease  in  Childhood,  July  3, 1979. 

6.  National  Institute  of  Neurological  and  Com- 
municative Disorders  and  Stroke,  "Survey  of 
the  Incidence,  Prevalence,  and  Cost  of  Inter- 
cranial  Neoplasm,"  1979. 

The  last  study  is  part  of  a  current  effort  by  the  Na- 
tional Institute  of  Neurological  and  Communicative 
Disorders  and  Stroke  to  link  research  priority  setting 
more  closely  to  identification  of  areas  of  greatest 
societal  need  and  projects  with  the  greatest  potential 
for  investment  return.  Thus,  it  is  perceived  as  con- 
tributing to  an  effort  to  produce  more  "cost-effec- 
tive" research  (480).  Similarly,  the  cost-of-dialysis 
studies  of  the  National  Institute  of  Arthritis,  Metabo- 


lism, and  Digestive  Diseases  are  perceived  as  having 
a  cost-effectiveness  element  because  they  underline 
the  different  costs  and  complications  of  various 
modes  of  dialysis. 

Each  institute  of  NIH  seeks  to  reduce  the  morbidity 
and  mortality  associated  with  the  disease  under  its 
mandate  and  considers  research  directions  that  are 
intended  to  reduce  the  human  and  economic  toll  of 
the  diseases.  Cost  considerations  have  occasionally 
been  considered  in  NIH  consensus  development  con- 
ferences. Although  their  primary  focus  is  on  "clini- 
cally relevant  research  and  the  readiness  of  certain 
important  findings  to  be  put  to  use  in  health  or  reg- 
ulation," these  conferences  may  also  consider  social, 
ethical,  legal,  and  cost  implications.  In  general,  how- 
ever, they  consider  these  implications  only  as  neces- 
sary extensions  of  their  scientific  and  technical  eval- 
uations of  specific  technologies.  The  principal  re- 
sponsibility for  examining  these  "nontechnical" 
issues  rests  with  the  National  Center  for  Health  Care 
Technology  (NCHCT),  which  is  discussed  in  a  sepa- 
rate section  of  this  appendix.  The  consensus  devel- 
opment conferences  sponsored  by  NIH  serve  only  as 
the  forum  for  the  issues'  presentation.  Consensus  de- 
velopment conferences  to  date  have  addressed  a  wide 
range  of  subjects,  including  mammography,  dental 
implants,  and  electronic  fetal  monitoring. 

In  addition,  three  NIH  institutes  have  produced 
more  explicit  cost-benefit  work.  One,  the  National 
Heart,  Lung,  and  Blood  Institute  (NHLBI)  has  con- 
ducted several  cost-benefit  studies.  NHLBI's  first  ex- 
perience with  CBA  was  in  a  1973  assessment  of  heart 
transplant  surgery  and  artificial  heart  development 
and  use.  As  part  of  its  long-term  program  of  examin- 
ing mechanical  circulatory  technology,  NHLBI  issued 
a  report  on  the  medical,  ethical,  legal,  financial,  and 
social  implications  of  artificial  heart  implantation 
(442).  This  assessment  did  not  include  a  formal  CBA, 
but  did  incorporate  the  concept  by  providing  a  de- 
tailed listing  of  the  costs,  as  well  as  the  medical  and 
social  benefits,  of  the  artificial  heart.  The  report's  ex- 
amination of  the  cost  of  the  device  probably  proved 
less  important  than  its  discussion  of  the  formidable 
engineering  obstacles  to  development,  however, 
since  it  was  these  obstacles  that  played  a  major  role 
in  discouraging  greater  investment  of  resources  by 
the  institute  in  the  totally  implantable  artificial  heart 
program  (514). 

In  1972,  NHLBI's  mandate  was  broadened  to  in- 
clude prevention,  education,  and  control  responsibil- 
ities. Subsequently,  the  National  High  Blood  Pres- 
sure Education  Branch  and  the  National  High  Blood 
Pressure  Demonstration  Program  were  created  to 
work  with  Government  and  private  agencies  in  an  ef- 
fort to  increase  an  awareness  of  the  dangers  of  hyper- 
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tension  and  to  encourage  the  development  of  effec- 
tive treatment.  Both  of  these  programs  have  utilized 
information  produced  by  the  cost-benefit  studies  to 
help  convince  State  and  local  health  agencies  and  pri- 
vate organizations  of  the  value  of  instituting  hyper- 
tension control  programs  (611,614). 

In  1975,  NHLBI's  National  High  Blood  Pressure 
Education  Program  conducted  a  CBA  of  a  model  hy- 
pertension control  program  in  order  to  produce  a 
model  for  analyzing  the  costs  and  benefits  of  a  na- 
tional hypertension  treatment  program.  This  study 
was  recently  updated  (441)  through  the  incorpora- 
tion of  new  data  and  the  development  of  a  computer 
program  which  tests  several  of  the  major  assump- 
tions made  in  the  1975  report  (577).  NHLBI  has  used 
information  from  this  and  the  earlier  analysis  to  en- 
courage effective  treatment  programs. 

Further,  the  National  Institute  of  Dental  Research 
(NIDR)  has  produced  two  cost-benefit  studies. 
NIDR's  National  Caries  Program  was  established  by 
Congress  in  1971  to  sponsor  R&D  activities  directed 
toward  ultimately  reducing  the  incidence  of  dental 
disease  in  the  American  public.  In  carrying  out  this 
mission,  program  officials  have  used  CBAs  to  assist 
them  in  their  long-range  planning  of  R&D  investment 
decisions  (83,522).  In  1974,  NIDR  contracted  for  the 
development  of  a  computer-based  model  for  predict- 
ing the  long-term  "net  social  value"  of  the  use  of  pre- 
ventive procedures  (462).  The  results  of  the  study 
were  expressed  in  terms  of  the  dollar  savings  pro- 
duced by  dental  care  treatment  on  both  a  nationwide 
and  regional  basis.  In  1978,  an  internal  NIDR  staff 
report  assessed  the  costs  and  benefits  of  a  specific 
caries  prevention  treatment,  pit  and  fissure  sealants 
(72). 

Both  of  these  NIDR  cost-benefit  studies  contrib- 
uted to  a  decision  to  halt  investment  in  occlusal  seal- 
ant clinical  trials  until  a  sealant  and  a  system  of  deliv- 
ery could  be  developed  that  would  significantly  re- 
duce the  clinical  time  required  in  sealant  application 
(523).  Examination  of  the  "net  social  value"  model 
also  contributed  to  program  decisions  to  increase 
funding  of  efforts  aimed  at  reducing  the  high  carcino- 
genicity of  American  diets  and  projects  directed 
toward  development  of  fluoride-based  regimens  for 
use  in  nonfluoride  areas  (523). 

Another  study  employing  the  cost-benefit  concept 
has  been  funded  by  the  National  Cancer  Institute 
(NCI).  NCI  originally  contracted  with  the  Blue 
Cross/Blue  Shield  Associations  to  develop  a  model 
prepaid  health  service  benefit  package  for  a  cancer- 
screening  program  (435,436).  Although  originally  the 
contents  of  the  benefit  package  were  to  be  deter- 
mined through  the  consensus  of  a  panel  of  experts, 
the  screening  program  that  eventually  was  designed 


was  largely  determined  by  the  use  of  cost-effective- 
ness methods  (168). 

Cost-benefit  considerations  are  often  present  in  the 
various  NIH  institutes'  clinical  trial  deliberations. 
Although  the  evaluation  of  benefits  is  largely  qual- 
itative, judgments  about  whether  to  undertake  a  clin- 
ical trial  in  order  to  test  a  given  therapy  often  broad- 
ly consider  costs  and  benefits.  Perhaps  the  most  sys- 
tematic and  formal  example  of  this  process  is  pro- 
vided by  NCI's  method  of  selecting  new  drugs  for 
clinical  trials.  Within  NCI,  a  large  staff  committee 
called  the  "Decision  Network  Group"  engages  in  a 
formal  evaluation  of  costs  and  potential  benefits.  A 
staff  member  describing  the  process  observed  (542): 
In  making  the  selection  of  specific  drugs  to  develop 
toward  clinical  trial,  we  consider  the  relative  biologi- 
cal activity  of  the  drug  in  comparison  with  the  cost  of 
development,  including  large-scale  production.  Thus, 
if  the  drug  represents  a  new  type  of  chemical  class 
which  is  highly  active  and  relatively  inexpensive,  we, 
of  course,  have  no  difficulty  in  reaching  a  decision.  If, 
however,  the  new  drug  is  of  relatively  marginal  bene- 
fit in  the  experimental  systems  we  utilize,  and,  in  addi- 
tion, it  is  estimated  to  cost  a  great  deal  of  money  to 
produce  and  develop,  we  very  seriously  consider  the 
advisability  of  proceeding. 

Another  example  of  this  type  of  cost  consideration 
is  a  new  drug  related  to  one  previously  developed.  If 
the  original  drug  was  very  expensive  and  the  new  ana- 
log is  considerably  cheaper  and  of  equivalent  activity, 
we  would  very  likely  decide  to  switch  to  the  new  drug. 

Cost  has  a  major  role  in  the  development  and  de- 
sign of  the  approaches  we  utilize  for  screening  new 
potential  antitumor  drugs  in  experimental  systems. 
Thus,  in  developing  the  panel  of  experimental  systems 
currently  utilized,  we  realized  that  it  was  impossible 
financially  to  carry  out  the  testing  in  what  might  be 
considered  the  ideal  scientific  manner,  namely,  to  test 
all  materials  in  the  complete  battery  of  experimental 
systems.  Since  that  approach  was  not  financially  fea- 
sible, we  developed  a  pre-screening  approach,  so  that 
the  large  number  of  compounds  being  tested  would  be 
evaluated  in  a  relatively  sensitive  system  first,  and 
then  further  evaluated  in  the  panel  of  experimental 
tumors  based  primarily  on  activity  in  that  pre-screen. 

Another  example  of  cost  analysis  involves  the  large 
animal  preclinical  toxicology  studies  carried  out  on  a 
drug  being  developed  for  clinical  trial.  Our  standard 
protocol  for  studies  requires  about  9-10  months  and 
costs  on  the  order  of  $120,000.  We  experimented  with 
one  very  high  priority  drug  to  determine  whether 
these  experiments  could  be  carried  out  in  a  shorter 
period  of  time.  We  found,  indeed,  that  this  could  be 
done,  but  unfortunately,  it  required  approximately 
twice  as  much  money  to  carry  out  those  experiments, 
since  more  tests  had  to  be  done  simultaneously  with 
some  wasted  effort  in  order  to  save  time.  Thus,  it  was 
decided  that  we  would  not  routinely  evaluate  drugs  in 
that  manner  since  it  was  so  expensive,  but  reserve 
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such  alternate  procedures  for  those  very  rare  com- 
pounds that  are  considered  of  such  high  priority  that 
they  must  be  moved  as  rapidly  as  possible  regardless 
of  the  cost  .  .  . 

In  summary,  although  we  cannot  carry  out  what 
one  might  like  to  see  as  completely  quantitative  cost 
benefit  analyses,  cost  considerations  have  always 
been  and  remain  a  vital  part  of  our  everyday  life  in 
making  scientific  decisions  and  in  attempting  to  devel- 
op new  and  better  modes  of  therapy  in  the  shortest 
amount  of  time  with  the  funds  available  to  use. 
Although  other  NIH  institutes  utilize  a  less  formal 
selection  process  than  NCI,  cost-benefit  considera- 
tions often  enter  into  their  decisions  concerning  the 
need  for  clinical  trials.  A  staff  member  of  NHLBI,  for 
example,  cited  three  instances  in  which  cost  consid- 
erations played  a  part  in  such  decisions3  (577). 

A  number  of  the  Institute's  clinical  trials  were  un- 
dertaken for  reasons  very  much  concerned  with  the 
potential  benefits  to  be  accrued  from  a  careful  testing 
of  a  given  therapy  or  preventive  regimen.  In  particu- 
lar, the  coronary  artery  surgery  study  (CASS)  is  to  de- 
termine the  efficacy  of  surgical  vs.  medical  interven- 
tion for  coronary  artery  disease  .  .  .  Coronary  artery 
surgery  (by-pass  surgery)  is  an  extremely  expensive 
operation  (between  $10,000  and  $20,000  per  case)  that 
may  be  able  to  be  treated  as  efficaciously  and  at  con- 
siderably less  cost  through  non-surgical  means.  In  ar- 
riving at  the  decision  to  undertake  CASS,  the  Institute 
weighed  the  potential  benefits  of  the  information  to  be 
derived  against  the  considerable  cost  of  the  study.  In 
similar  fashion,  the  Institute  considered  a  trial  related 
to  mild  hypertension  treatment,  but  decided  that  the 
Institute's  current  portfolio  of  clinical  trials  related  to 
hypertension  would  provide  much  the  same  informa- 
tion as  a  new  trial.  It  was  calculated  that  such  a  trial 
was  not  currently  warranted.  The  Extracorporeal 
Membrane  Oxygenator  (ECMO)  and  Intermittent 
Positive  Pressure  Breathing  (IPPB)  trials  are  two  addi- 
tional NHLBI  supported  trials  concerned  with  deter- 
mining the  effectiveness  of  costly,  unvalidated  ther- 
apies. 

Food  and  Drug  Administration 

FDA  is  responsible  for  monitoring  the  safety  of 
foods  and  cosmetics  and  evaluating  the  safety  and  ef- 
ficacy of  drugs  and  medical  devices,  food,  feed,  and 
color  additives.  With  the  exception  of  several  studies 
performed  by  the  Bureau  of  Radiological  Health 
(BRH),  FDA's  experience  with  CEA  is  limited  to  eight 
analyses  conducted  in  response  to  regulatory  initia- 
tives.4 

'For  a  discussion  of  NHLBI  clinical  trial  planning  and  decisionmaking, 
see  R.  I.  Levy  and  E.  J.  Sondik,  "Decision-Making  in  Planning  Large-Scale 
Comparative  Studies,"  and  R.  I.  Levy  and  E.  J.  Sondik,  "Initiating  Large- 
Scale  Clinical  Trials,"  1978. 

4For  other  regulatory  agencies'  experience  with  these  requirements,  see  T. 
B.  Clark,  "Its  Still  No  Bureaucratic  Revolution,  But  Regulatory  Reform  Has 
Foothold,"  National  Journal,  Sept.  29,  1979  (97). 


On  March  23,  1978,  Executive  Order  12044  man- 
dated that  Government  agencies  examine  the  costs 
and  benefits  of  major  proposed  regulations  (i.e.,  reg- 
ulations having  an  annual  impact  on  the  economy  of 
"$100  million  or  more  or  [causing]  a  major  increase 
in  costs  or  prices  for  individual  industries,  levels  of 
government  or  geographic  regions").  This  mandate 
was  built  upon  an  earlier  executive  order  issued  by 
President  Ford  which  required  an  "inflation  impact" 
analysis  of  major  proposed  regulations  in  order  to 
assure  that  the  private  sector  not  be  burdened  by  un- 
justified costs.5 

Under  these  mandates,  FDA  has  conducted  eco- 
nomic analyses  of  eight  proposed  initiatives  (218): 

1.  Food  and  Drug  Administration,  Office  of  Plan- 
ning and  Evaluation,  Office  of  the  Commis- 
sioner, "Inflation  Impact  Statement  of  Pro- 
posed Rulemaking:  Certain  Fluorocarbons  and 
Chlorofluorocarbons  in  Food,  Food  Additives, 
Drug,  Animal  Food,  Animal  Drug,  Cosmetic 
and  Medical  Device  Products  as  Propellants  in 
Self-Pressurized  Containers,  Prohibition  on 
Use,"  May  1977. 

2.  Food  and  Drug  Administration,  Office  of  Plan- 
ning and  Evaluation,  Office  of  the  Commis- 
sioner, "Inflation  Impact  Statement  of  Pro- 
posed Rulemaking:  Nitrofuran  (S-Nitro)  Com- 
pounds," May  1976. 

3.  Food  and  Drug  Administration,  Office  of  Plan- 
ning and  Evaluation,  Office  of  the  Commis- 
sioner, "Inflation  Impact  Statement  of  Pro- 
posed Rulemaking:  Diethylstilbestrol,"  January 
1976. 

4.  F.  H.  Dworkin,  Economics  Staff,  Office  of 
Planning  and  Evaluation,  Food  and  Drug  Ad- 
ministration, "An  Analysis  of  Economic  Im- 
pact of  the  Drug  Regulation  Reform  Act  of 
1978,"  November  1978. 

5.  F.  H.  Dworkin,  Economics  Staff,  Office  of 
Planning  and  Evaluation,  Food  and  Drug  Ad- 
ministration, "Chemical  Compound  in  Food- 
Producing  Animals:  Criteria  and  Procedures 
for  Evaluating  Assays  for  Carcinogenic  Resi- 
dues," March  1979. 

6.  Food  and  Drug  Administration,  Office  of  Plan- 
ning and  Evaluation,  Office  of  the  Commis- 
sioner, "Inflation  Impact  Statement  of  Pro- 
posed Rulemaking:  Saccharin  and  Its  Salts  in 
Foods,  Drugs,  Animal  Feeds,  Animal  Drugs, 
Cosmetics,  and  as  a  Food  Additive,  Prohibition 
of  Uses,"  April  1977. 

7.  P.  F.  Lewis,  Environmental  and  Economic  Im- 
pact Staff,  Bureau  of  Foods,  Food  and  Drug 

5This  earlier  executive  order  was  Executive  Order  11821,  superseded  by 
Executive  Order  11949,  1977. 
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Administration,  "Regulatory  Analysis  for  Final 
Regulation  for  Reduction  of  Temporary  Toler- 
ances for  Polychlorinated  Biphenyls  in  Food," 
June  27, 1977. 
8.  F.  H.  Dworkin,  et  al.,  Economics  Staff,  Office 
of  Planning  and  Evaluation,  Food  and  Drug 
Administration,  "Draft  Regulatory  Analysis: 
Patient  Labeling  Requirements  for  Prescription 
Drug  Products,"  June  1979. 
In  most  of  these  economic  analyses,  however,  FDA 
was  only  able  to  list  benefits.  In  general,  it  was  not 
able  to  quantify  them  owing  to  a  lack  of  adequate 
data,  methodological  problems,  and  difficulty  in  pro- 
jecting behavior  precipitated  by  proposed  manufac- 
turing controls  (43,646).  Further,  when  an  analysis 
involves  a  carcinogenic  food  additive,  as  it  did  in 
three  instances,6  FDA  is  legally  forbidden  to  weigh 
costs  and  benefits  in  making  a  decision.  According  to 
the  "Delaney  clause"  of  the  Food,  Drug,  and  Cosmet- 
ic Act  of  1938, 7  if  a  food  additive  is  found  to  be  car- 
cinogenic, FDA  must  ban  it.8  Nevertheless,  FDA 
seems  to  have  found  economic  analyses  generally 
useful,  and  the  consideration  of  specific  benefits 
probably  has  contributed  to  the  modification  of  some 
provisions  of  several  proposed  regulations  (43). 9  10 

One  of  FDA's  component  bureaus,  BRH,  has  used 
economic  analyses  and  CBAs  more  extensively  than 
other  component  bureaus.  BRH  is  responsible  for 
protecting  the  public  from  unnecessary  exposure  to 
radiation  and  ensuring  that  radiation  is  used  safely 
and  efficaciously.  In  1977,  BRH  conducted  an  "Eco- 
nomic Impact  Assessment  of  the  Proposed  Perform- 
ance Standard  for  Sunlamp  Products,"  January  1977. 
Because  of  the  limited  projected  impact  of  this  regula- 
tion, BRH  assessed  only  costs. 

In  April  of  1977,  however,  BRH  used  the  cost- 
benefit  concept  more  specifically,  to  justify  budget 
requests  for  expanding  the  Bureau's  effort  to  reduce 
the  unnecessary  use  of  X-rays.  Thus,  the  fiscal  year 
1979  Preliminary  Budget  Justification  stated  (217): 
The  proposed  national  X-ray  system  will  cost  an  ad- 
ditional $7.7  million  contract  dollars  and  181  posi- 
tions. This  gives  a  benefit/cost  ratio  ranging  from  32 
to  1  to  70  to  1  for  the  20-50  percent  possible  reductions 
in  the  genetically  significant  dose  alone. 

"The  inflation  impact  statements  for  nitrofuran,  diethylstilbestrol,  and 
saccharin. 

'See  Office  of  Technology  Assessment,  U.S.  Congress,  Cancer  Testing 
Technology  and  Saccharin.  October  1977  (466). 

This  legislation  takes  precedence  over  Executive  Order  12044. 

'The  cost  of  releasing  safety  and  efficacy  data  on  new  drugs,  revealed  by 
the  analysis  of  the  1978  Drug  Regulatory  Reform  Act,  for  example,  contrib- 
uted to  the  dropping  of  the  draft  requirement  that  such  information  be  re- 
leased. 

'"Another  case  in  which  economic  analysis  contributed  to  a  modification 
of  policy  is  FDA's  action  on  antibiotics  and  feed:  "Combined  Economic  Im- 
pact Assessment  of  Series  of  Proposals  Regarding  Penicillin/Tetracycline- 
Containing  Premixes."  undated. 


This  cost-benefit  argument  was  the  "driving  force" 
(341)  behind  the  development  of  a  cost  containment 
plan:  "A  Proposed  FDA  Program  To  Reduce  Unnec- 
essary Patient  Exposure  From  Diagnostic  X-Rays: 
Cost  Containment  Considerations"  August  1978.  In 
this  plan,  CBA  was  used  to  demonstrate  the  cost  sav- 
ings produced  by  reducing  unnecessary  X-rays. 

Another  CBA  performed  by  BRH  was  "The  Diag- 
nostic X-Ray  Equipment  Performance  Standard  and 
the  Policy  on  Assembly  and  Reassembly,"  November 
1978.  This  study  represented  a  refinement  and  elabo- 
ration of  an  earlier  attempt  (216)  to  examine  the  costs 
and  benefits  of  the  X-ray  equipment  standard  in  an 
environmental  assessment  report  in  1974.  The  results 
of  this  updated  study  contributed  to  FDA's  decision 
to  amend  its  policy  on  the  assembly  and  reassembly 
of  diagnostic  X-ray  equipment  and  revoke  two  reg- 
ulatory provisions  which  were  shown  not  to  be  cost 
beneficial  (215,340). 

Center  for  Disease  Control 

CDC  is  responsible  for  monitoring,  controlling, 
and  reducing  the  incidence  of  preventable  diseases 
and  conditions.  Over  the  years,  CDC  has  conducted 
a  number  of  assessments  of  the  costs  of  illnesses  and 
treatment  methods,  as  well  as  cost-benefit  studies.  In 
1966,  CDC  involvement  in  the  HEW  program  anal- 
yses of  the  costs  and  benefits  of  tuberculosis  and 
syphilis  control  promoted  staff  interest  in  examining 
the  economic  impact  of  disease.11  Since  then,  CDC 
has  often  used  economic  analysis  to  supplement  the 
Center's  traditional  public  health  perspective  by 
allowing  the  costs  of  a  disease  to  be  considered  along 
with  morbidity  and  mortality  statistics.  Since  much 
of  CDC's  work  involves  providing  information  to 
State  and  local  public  health  officials  who  must  be 
concerned  with  budgets,  economic  analysis  has 
proved  particularly  helpful  in  supporting  the  value  of 
suggested  disease  prevention  control  programs  (32, 
601).  A  CDC  official  noted: 

[These  analyses]  enable  the  cost  of  disease  control 
efforts  and  investigations  to  be  placed  in  perspective. 
For  vector-borne  diseases  (dengue,  equine  encephali- 
tis), such  analyses  make  it  easier  to  justify  costly 
spraying  and  clean-up  efforts. 

The  economic  impact  studies  conducted  by  CDC 
staff  include: 

1.  D.  J.  Sencer  and  N.  W.  Axnick,  "Utilization  of 
Cost  Benefit  Analysis  in  Planning  Prevention 
Programs,"  Acta  Medica  Scandinavia,  Supple- 
ment 576,  1975.  [This  study  reports  on  the 

"See  the  section  in  this  appendix  on  the  Office  of  the  Assistant  Secretary 
for  Planning  and  Evaluation,  DHHS.  Also,  for  a  discussion  of  CDC's  anal- 
yses of  syphilis  and  tuberculosis  control,  see  R.  N.  Grosse,  "Cost-Benefit 
Analysis  of  Health  Services,"  Ann.  Am.  Ac.  1972  (266). 
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costs  of  hepatitis,  hospital  infections,  and  the 
1964-65  rubella  epidemic.  ] 

2.  D.  D.  Tolsma  and  J.  A.  Bryan,  "The  Economic 
Impact  of  Viral  Hepatitis  in  the  United  States," 
Public  Health  Reports  91:349,  1979. 

3.  S.  D.  Von  Allmen,  R.  H.  Lopez-Correa,  J.  P. 
Woodall,  D.  M.  Morens,  J.  Chiniboga,  and  A. 
Casta- Valez,  "Epidemic  Dengue  Fever  in  Puer- 
to Rico,  1977:  A  Cost  Analysis,"  American 
Journal  of  Tropical  Medicine,  November  1979. 

4.  E.  L.  Baker,  Jr.,  W.  Peterson,  S.  D.  Von  All- 
men,  and  J.  Fleming,  "Economic  Impact  of  a 
Community-Wide  Waterborne  Outbreak  of 
Gastrointestinal  Illness." 

5.  M.  L.  Cohen,  R.  E.  Fontaine,  R.  A.  Hollard,  S. 
D.  Von  Allmen,  T.  M.  Vernon,  and  E.  J.  Gan- 
garosa,  "An  Assessment  of  Patient-Related 
Costs  in  an  Outbreak  of  Salmonellosis." 

6.  D.  D.  Tolsma  and  J.  D.  Millar,  "Costs  and 
Accomplishments  of  the  United  States  Nation- 
al Influenza  Immunization  Program,  1976," 
Eighth  International  Scientific  Meeting  of  the 
International  Epidemiologic  Association  Sym- 
posium on  Epidemiologic  Aspects  of  Cost-Ef- 
fectiveness Studies,  San  Juan,  Puerto  Rico, 
Sept.  19, 1977. 

7.  R.  C.  Rendtorff,  J.  C.  Curran,  R.  W.  Chandler, 
W.  L.  Wiser,  and  H.  Robinson,  "Economic 
Consequences  of  Gonorrhea  in  Women:  Ex- 
periences From  an  Urban  Hospital,"  Journal  of 
the  American  Venereal  Disease  Association, 
1(1),  September  1974. 

8.  S.  D.  Von  Allmen,  W.  Cates,  Jr.,  K.  F.  Schulz, 
D.  A.  Grimes,  and  C.  W.  Tyler,  Jr.,  "Cost  of 
Treating  Abortion-Related  Complications," 
Family  Planning  Perspectives,  9(6),  Novem- 
ber/December 1977. 

9.  J.  L.  Bradley,  "The  Cost  of  Diphtheria  Epidem- 
ic in  San  Antonio,  Texas,  1970-71,"  paper  pre- 
sented at  the  American  Public  Health  Associa- 
tion Meeting,  San  Francisco,  Nov.  7,  1973. 

10.  P.  M.  Schwab,  "Economic  Cost  of  St.  Louis  En- 
cephalitis Epidemic  in  Dallas,  Texas,  1966," 
Public  Health  Reports,  83(10) :860,  October 
1968. 

11.  B.  S.  Levy,  "The  Economic  Impact  of  a  Food- 
Borne  Salmonellosis  Study,"  Journal  of  Ameri- 
can Medical  Association,  230:1281,  Dec.  2, 
1974. 

12.  N.  W.  Axnick,  "Cost  of  1968-69  Influenza  Epi- 
demic," paper  presented  to  the  Surgeon  Gener- 
al's Advisory  Committee  on  Immunization 
Practice,  April  1970. 

13.  S.  D.  Von  Allmen,  R.  H.  Lopez-Correa,  J.  P. 
Woodall,  D.  M.  Morens,  J.  Chiniboge,  and  A. 


Casta- Valez,  "Epidemic  Dengue  Fever  in  Puer- 
to Rico,  1977:  A  Cost  Analysis,"  American 
Journal  of  Tropical  Medicine,  November  1979. 

14.  E.  L.  Baker,  Jr.,  W.  Peterson,  S.  D.  Von  All- 
men,  and  J.  Fleming,  "Economic  Impact  of  a 
Community-Wide  Waterborne  Outbreak  of 
Gastrointestinal  Illness." 

15.  M.  L.  Cohen,  R.  E.  Fontaine,  R.  A.  Pullard,  S. 
D.  Von  Allmen,  T.  M.  Vernon,  and  E.  J.  Gan- 
garosa,  "An  Assessment  of  Patient-Related 
Costs  in  an  Outbreak  of  Salmonellosis." 

16.  N.  W.  Axnick  and  J.  M.  Lane,  "Costs  Associ- 
ated With  the  Protection  of  the  United  States 
Against  Smallpox,"  World  Health  Organiza- 
tion Report,  March  1973. 

17.  P.  M.  Layde,  S.  D.  Von  Allmen,  and  G.  P. 
Oakley,  "Antenatal  Screening  for  Downs  Syn- 
drome: An  Economic  Viewpoint,"  September 
1979. 

18.  J.  J.  Witte  and  N.  W.  Axnick,  "The  Benefits 
From  10  Years  of  Measles  Immunization  in  the 
United  States,"  Public  Health  Reports,  90:3, 
May-June  1975. 

In  recent  years,  CDC  also  has  conducted  a  number 
of  cost-benefit  studies.  Several  CDC  officials  noted 
that  CBA  contributes  an  additional  dimension  to 
more  traditional  evaluations  of  morbidity  and  mor- 
tality, and  where  adequate  data  exist,  it  has  become 
logical  to  consider  this  information  an  evaluation.  In 
some  cases,  CBA  seems  to  provide  a  particularly 
meaningful  perspective,  because  humanitarian  con- 
cerns can  be  weighed  against  dollar  costs  and  the 
value  of  prevention  can  be  made  more  tangible  by 
describing  benefits  in  terms  of  averted  costs.  As  a 
consequence,  CBA  has  been  used  by  CDC  staff  to 
plan  and  evaluate  prevention  and  control  programs 
and  justify  investment  in  such  efforts  (32,325). 

Cost-benefit  assessments  conducted  for  the  CDC 
staff  include: 

1.  P.  M.  Layde,  S.  D.  Von  Allmen,  and  G.  P. 
Oakley,  "Maternal  Serum  Alpha-Fetoprotein 
Screening:  A  Cost  Benefit  Analysis,"  American 
Journal  of  Public  Health,  69(6):566,  June  1979. 

2.  P.  M.  Layde,  S.  D.  Von  Allmen,  and  G.  P. 
Oakley,  "Congenital  Hypothyroidism  Control 
Programs:  A  Cost-Benefit  Analysis,"  Journal  of 
the  American  Medical  Association,  24(21): 
2290,  May  1979. 

3.  P.  V.  Strange  and  A.  T.  Sumner,  "Predicting 
Treatment  Costs  and  Life  Expectancy  for  End- 
Stage  Renal  Disease,"  N.  Eng.  J.  Med.,  298: 
372,  Feb.  16, 1978. 

4.  D.  J.  Sencer  and  N.  W.  Axnick,  "Cost  Benefit 
Analyses,"  Proceedings  of  the  145th  Sympo- 
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sium  on  Vaccination  Against  Communicable 
Diseases,  Monaco,  March  1973. 

5.  J.  P.  Koplan,  S.  Schoenbaum,  M.  Weinstein, 
and  D.  Fraser,  "Pertussis  Vaccine:  An  Analysis 
of  Benefits,  Risks,  and  Costs,"  N.  Eng.  ].  Med., 
Oct.  25,  1979. 

6.  J.  K.  Paperfuss  and  B.  C.  Ejeldsled,  "Cost-Ben- 
efit Study  of  Selected  Interventions  in  Control 
and  Prevention  of  Tuberculosis  in  the  State  of 
Michigan,"  study  prepared  by  the  Michigan 
Division  of  Health  for  CDC,  December  1969. 

Alcohol,  Drug  Abuse,  and 
Mental  Health  Administration 

ADAMHA  has  major  responsibility  for  the  pre- 
vention and  treatment  of  mental  illness,  alcohol 
abuse,  and  drug  abuse.  One  of  ADAMHA's  three 
component  bureaus,  the  National  Institute  on  Drug 
Abuse  (NIDA),  has  produced  several  cost-of-illness 
and  cost-benefit  studies.  They  include: 

1.  Arthur  D.  Little,  "Social  Cost  of  Drug  Abuse," 
1976. 

2.  Macro  Systems,  Inc.,  "Unit  and  Episodic  Costs 
of  Drug  Abuse  Treatment:  A  Feasibility 
Study,"  May  1978. 

3.  Center  for  Social  Research  on  Drug  Abuse, 
'The  Assessment  of  Social  Cost  Savings," 
March  1978. 

4.  Research  Triangle  Institute,  "Management  Ef- 
fectiveness Measures  for  NIDA  Drug  Abuse 
Treatment  Programs:  Costs  to  Society,"  vol.  II, 
1977. 

These  studies  have  been  used  in  both  planning  and 
budget  justification.  The  cost  information  yielded  by 
the  analyses  has  provided  basic  data  for  CEA  CBA 
of  drug  abuse  treatment. 

In  1972,  the  U.S.  Bureau  of  Narcotics  and  Danger- 
ous Drugs  conducted  a  study  of  the  costs  and  benefits 
of  alternative  approaches  to  addiction  control  (386). 
A  study  funded  by  NIDA  in  1975  extended  this  idea 
by  developing  four  primary  measures  of  the  effec- 
tiveness of  drug  treatment  and  comparing  the  cost- 
effectiveness  of  five  treatment  modalities:  methadone 
maintenance,  therapeutic  community,  inpatient  de- 
toxification, outpatient  detoxification,  and  drug-free 
treatment  (535a). 

More  recent  studies  have  focused  on  the  develop- 
ment of  a  more  sophisticated  cost-effectiveness  and 
cost-benefit  ratios  for  drug  abuse  treatment  programs 
(504)  and  examination  of  the  applicability  of  CEA  to 
primary  drug  abuse  prevention  programs  (298,505). 
These  recent  NIDA  analyses  are  acknowledged  to 
have  serious  methodological  limitations  because  of 
inadequate  data  about  such  factors  as  the  total  num- 


ber of  drug  abusers,  the  relapse  rate  of  treated 
abusers,  the  relationship  between  drug  abuse  and  un- 
employment and  crime,  and  the  relationship  between 
patient  outcome  and  treatment  modality  (535). 

Nevertheless,  the  cost-benefit/cost-effectiveness 
studies  are  credited  with  providing  valuable  informa- 
tion for  program  evaluation  and  management  and 
NIDA  budget  justification  (360,520,534,535a).  The 
studies  have  also  confirmed  the  worth  of  allocating 
more  funds  to  the  less  traditional  and  expensive  treat- 
ment approaches  (i.e.,  of  a  funding  formula  which 
favors  outpatient  methadone  maintenance,  outpa- 
tient drug  detoxification,  and  outpatient  drug-free 
programs  rather  than  inpatient  detoxification  and 
residential  community  approaches).  In  addition,  they 
have  been  useful  in  confirming  the  benefits  produced 
by  the  Federal  investment  in  drug  abuse  treatment. 
As  an  official  of  NIDA  observed  (535a): 

.  .  .  Based  on  an  empirical  analysis  of  data,  we 
have  established  average  gains  per  patients  year  on 
performance  criteria  and  can  translate  these  gains  into 
cost  savings  benefits  to  society.  In  turn,  we  can  com- 
pare them  to  what  society  has  invested  in  treatment. 
The  conclusion  is  that  it  pays  to  invest  in  treatment. 
Unfortunately,  such  analyses  are  still  severely  lim- 
ited by  their  inability  to  specify  which  treatment  mo- 
dality tends  to  work  best  for  a  given  type  of  client.  In 
order  to  better  address  this  question,  NIDA  is  cur- 
rently attempting  to  collect  more  accurate  data  on 
drug  abuse  and  rehabilitation  patterns  through  the 
Drug  Abuse  Reporting  Program  (DARP)  and  the 
Treatment  Outcome  Prospective  Study  (TOPS). 

At  the  National  Institute  of  Mental  Health 
(NIMH),  there  are  two  potential  applications  of 
CEA  CBA  studies:  1)  in  measurement  of  service  pro- 
gram outcomes,  such  as  the  community  mental 
health  centers  (CMHCs)  that  receive  Federal  spon- 
sorship, and  2)  the  measurement  or  evaluation  of 
psychosocial  interventions.  It  is  important  to  keep 
these  two  quite  different  research  levels  separated. 
Depending  on  the  research  focus — program  out- 
comes or  psychosocial  treatment  outcomes — there 
will  be  differences  in  what  is  measured,  the  means  of 
measurement,  the  purpose  of  measurement,  the  uses 
of  the  information,  and  the  types  of  people  involved 
in  the  research. 

The  Psychotherapy  and  Behavioral  Intervention 
Section  of  NIMH's  Division  of  Extramural  Research 
Programs  is  responsible  for  funding  research  on  the 
effectiveness,  efficacy,  safety,  etc.,  of  psychosocial 
treatment  mechanisms  for  specific  mental  disorders. 
Although  one  or  two  of  the  NIMH-sponsored  psy- 
chotherapy outcome  studies  have  examined  variables 
related  to  cost  benefit  or  cost  effectiveness  (e.g.,  du- 
ration of  treatment  required),  most  such  studies  are 
not  intended  or  designed  to  address  cost  variables. 
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Traditionally,  psychotherapy  research  seeks  only  to 
elucidate  the  mechanisms  of  action  and  the  efficacy 
of  specific  treatments  for  specific  diagnostic  catego- 
ries of  mental  disorders.  The  Psychotherapy  and  Be- 
havioral Intervention  Section  is  currently  attempting 
to  encourage  the  use  of  a  range  of  standard  proce- 
dures for  the  measurement  of  change  in  psychother- 
apy in  the  hope  of  achieving  greater  comparability  in 
the  assessment  of  outcomes  across  various  studies. 
Cost-effectiveness  variables,  though,  are  even  more 
difficult  to  apprehend  and  often  are  not  useful  at  this 
level  of  research. 

The  Division  of  Biometry  and  Epidemiology  at 
NIMH  has  conducted  several  studies  of  the  services 
programs  and  episodes-of-illness  costs  of  patients 
treated  in  federally  sponsored  CMHCs.  Costs  and 
cost-effectiveness  criteria  appear  to  be  potentially  rel- 
evant for  use  in  evaluating  program  costs  and  exam- 
ining the  outcomes  of  delivering  mental  health  serv- 
ices at  the  community  level.  NIMH  has  funded  stud- 
ies under  contract  which  seek  to  advance  the  meth- 
odological framework  of  CBAs  as  applied  to 
CMHCs'  programs  (443).  NIMH  (or  the  Division) 
does  not  conduct  such  studies  itself  because  of  the 
limited  development  of  the  methods,  differences  in 
CMHCs'  clientele,  and  the  tendency  for  such  assess- 
ments to  be  subject  to  misrepresentation  and  to  be 
used  to  criticize  the  management  of  individual 
CMHCs.12 

NIMH  has  supported  training  courses  in  CBA 
through  its  staff  ..college,  and  through  the  contract 
mechanism,  is  supporting  several  State  governments 
in  the  development  of  management  systems  that  will 
provide  unit  cost  data  for  mental  health  service  sys- 
tems. A  few  States  (Colorado,  Oregon,13  and  Wash- 
ington) have  officials  who  are  working  with  NIMH 
to  develop  a  more  sophisticated  methodology  for  ex- 
amining CMHC  program  costs  and  outcomes. 

National  Center  for  Health  Care 
Technology 


(See  ch.  9  for  a  discussion  of  NCHCT.) 


"For  example,  see  D.  E.  Carter  and  F.  L.  Newman,  A  Client  Oriented 
System  of  Mental  Health  Service  Delivery  and  Program  Management:  A 
Workbook  and  Guide,  1976  (85);  J.  E.  Sorensen  and  D.  W.  Phillips,  "Cost 
Finding  and  Rate-Setting  for  Community  Mental  Health  Centers,"  1972 
(578);  and  W.  Hargeaves,  et  al..  Resource  Materials  for  Community  Mental 
Health  Program  Evaluation. 

"See  G.  Brodsky  and  D.  A.  Bigelow,  The  Oregon  Program  Impact  Moni- 
toring System,  1978  (70). 


National  Center  for  Health  Services 
Research 

NCHSR  funds  a  number  of  CEA  and  CBA  studies 
as  part  of  its  services  research  function  (439).  It  is 
currently  the  major  supporter  of  CEAs  and  CBAs  in 
the  Federal  health  care  research  system.  Two  of 
NCHSR's  research  priority  areas — health  care  costs 
and  cost  containment,  and  planning  and  regulation — 
specifically  call  for  research  using  CEA  studies  to  ex- 
amine the  issues  in  these  priority  areas  (438). 

The  types  of  economic  analyses  NCHSR  supports 
range  from  what  could  be  considered  traditional 
CEAs  or  CBAs  to  analyses  that  focus  on  either  costs 
or  outcomes  of  a  given  technology  or  program.  It  is 
difficult  to  characterize  the  range  of  economic  studies 
NCHSR  supports.  The  studies  range  from  rigorous 
examinations  of  specific  health  care  topics  to  broader 
studies  of  more  complex  health  care  issues.  CEA  or 
CBA  may  be  the  focus  of  a  grant  or  may  be  incorpo- 
rated into  larger,  more  global  assessments.  Never- 
theless, NCHSR  is  a  significant  source  of  support  for 
these  types  of  techniques.  Some  of  the  more  recent 
CEA  and  CBAs  that  have  received  support  from 
NCHSR  are  listed  below  (439) : 

1.  "Impact  of  Ophthalmic  Technicians  on  Outpa- 
tient Care,"  HS  03647. 

2.  "Improving  Clinical  Decision  Making  in  Arth- 
ritis," HS  03115. 

3.  "Prediction  of  Cost  Effective  Stroke  Rehabilita- 
tion," HS  03693. 

4.  "Controlled  Trial  of  a  Quality  Assurance 
Mechanism,"  HS  02485. 

5.  "Evaluation  of  Breast  Cancer  Detection  Strate- 
gies," HS  03256. 

6.  "Cost   Effective    Strategies    in  Ambulatory 
Care,"  HS  02063. 

7.  "A  Guide  to  Investment  Criteria  for  Critical 
Care  Units,"  HS  03569. 

National  Center  for  Health  Statistics 

NCHS  is  an  agency  within  the  Office  of  the  Assist- 
ant Secretary  for  Health  of  DHHS.  It  is  one  of  the 
principal  health  services  research  agencies  of  the  Fed- 
eral Government.  Health  statistics  activities,  which 
eventually  were  formalized  and  combined  by  the  cre- 
ation of  NCHS,  were  authorized  by  Congress  in  1946 
(vital  statistics)  and  in  1956  (National  Health  Sur- 
vey). NCHS  was  formed  in  1960  and  has  played  a 
major  role  in  the  development  of  national  health  sta- 
tistics policy  and  programs.  Under  its  current 
mandate — the  Health  Services  Research,  Health  Sta- 
tistics, and  Medical  Libraries  Act  of  1974  (Public  Law 
93-353)— NCHS  is  responsible  for  collecting  and  dis- 
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seminating  health  data  including  information  on  the 
costs  of  illness,  health  care,  and  health  financing. 

The  importance  of  data  collected  by  NCHS  cannot 
be  overemphasized.  Information  such  as  that  con- 
cerning the  incidence  and  prevalence  rates  of  dis- 
eases, natural  history  of  disease,  medical  care  utiliza- 
tion, workloss,  surgical  rates,  and  premature  mortal- 
ity is  crucial  to  CEA/CBA.  Although  NCHS  does  not 
conduct  full  cost-effectiveness  studies,  it  has  con- 
ducted cost-of-illness  studies,  and  these  have  been 
used  in  other  agencies'  CEA/CBAs.  Currently, 
NCHS  staff  are  coordinating  an  interagency  Public 
Health  Service  (PHS)  Cost  of  Illness  Committee 
which  is  examining  the  state-of-the-art  of  estimating 
costs  of  illness  and  disability.  As  noted  by  an  NCHS 
official,  it  has  often  proved  impossible  to  compare 
the  results  of  cost-of-illness  studies  conducted  by 
NCHS  and  other  DHHS  agencies  (297): 

Frequently  .  .  .  the  assumptions,  methods  and  data 

employed  vary  to  such  an  extent  that  estimates  from 

two  different  studies  differ  markedly,  and  the  costs  of 

several  illnesses  cannot  be  compared. 

In  an  attempt  to  address  this  problem,  the  PHS 
Cost  of  Illness  Committee  will  recommend  a  set  of 
guidelines  for  future  PHS  studies. 

Health  Resources  Administration 

HRA  is  responsible  for  improving  the  national  ca- 
pacity to  develop  and  effectively  use  health  re- 
sources. Its  Office  of  Program  Planning  and  Evalua- 
tion has  sponsored  conferences  which  produced  two 
studies  focusing  on  cost-effectiveness  methodology: 

1.  S.  O.  Schweitzer,  "The  Economics  of  the  Early 
Diagnosis  of  Disease."  [This  article  presents  a 
methodological  framework  for  the  evaluation 
of  the  cost-effectiveness  of  early  diagnostic 
tests.] 

2.  H.  A.  Foley,  D.  B.  Fisherman,  and  F.  Kalibat, 
"The  Cost-Effectiveness  of  a  Mental  Health 
Center:  An  Experiment,"  in  Evaluation  in 
Health  Services  Delivery,  edited  by  R.  Yaffe 
and  D.  Zalkind,  proceedings  of  an  Engineering 
Foundation  Conference,  South  Berwick, 
Maine,  Aug.  19-24,  1973. 

The  Bureau  of  Health  Manpower  (BHM),  one  of 
HRA's  component  agencies,  has  also  produced  one 
cost-effectiveness  study: 

1.  N.  Doherty,  "Study  to  Determine  the  Cost  and 
Effectiveness  of  Different  Practice  Modes  of 
Dental  Care  for  Children,"  February  1977. 
Another  HRA  agency,  the  Bureau  of  Health  Plan- 
ning (BHP),  has  sponsored  two  cost-benefit  studies: 
1.  Arthur  D.  Little,  "Introduction  to  Cost-Benefit 
Analysis  Applied  to  New  Health  Technol- 
ogies," December  1977. 


2.  Stanford  Research  Institute,  "Decision  Making 
Methodology  for  Health  Care  Services,"  study 
prepared  for  the  Region  IX  (San  Francisco)  Of- 
fice of  DHEW,  December  1979. 
Title  V  of  the  National  Health  Planning  and  Re- 
sources Development  Act  of  1974   (Public  Law 
93-641)  suggests  that  local  health  planning  agencies 
(health  systems  agencies  (HSAs))  should  consider  the 
costs  and  benefits  of  projects  (sec.  1514(3)): 

In  establishing  the  Annual  Implementation  Plan, 
the  agency  shall  give  priority  to  those  objectives 
which  will  maximally  improve  the  health  of  the  resi- 
dents of  the  area,  as  determined  on  the  basis  of  the  re- 
lation of  the  cost  of  attaining  such  objectives  to  their 
benefits,  and  which  are  fitted  to  the  special  needs  of 
the  area. 

But  most  HSAs  do  not  appear  to  use  CBA.14  The  two 
guidebooks  to  cost-benefit  methodology  which  BHP 
has  funded  constitute  only  a  small  segment  of  the 
technical  assistance  literature  which  is  being  distrib- 
uted to  HSAs. 

Health  Services  Administration 

The  Health  Services  Administration  is  responsible 
for  Federal  programs  that  provide  health  care  serv- 
ices to  specific  populations.  It  has  sponsored  a  num- 
ber of  studies  that  deal  with  cost-benefit  considera- 
tions: 

1.  Abt  Associates,  Inc.,  "Analysis  of  Benefits  and 
Costs  of  Ambulatory  Care  Services  Offered  by 
the  Bureau  of  Community  Health  Services," 
March  1975. 

2.  Geomet,  Inc.,  "Comparison  of  Cost-Per-Day  in 
PHS  Hospitals:  An  Approach  to  Service  and 
Cost  Comparisons,"  September  1975. 

3.  Health  Services  Administration/OPEL  and 
Geomet,  Inc.  (A.  Dobson  and  H.  L.  Heaton), 
"An  Economic  Analysis  of  HSA  Program  Po- 
tential, 1976-1981,"  report  No.  OPEL  75-4, 
May  1975. 

4.  Health  Services  Administration/OPEL  and 
Geomet,  Inc.  (I.  Enger  and  N.  Pindus),  "Cost 
and  Utilization  Evaluation  Migrant  Hospital 
Program,"  report  No.  OPEL  77-1,  January 
1977. 

5.  Health  Services  Administration/OPEL  and 
Geomet,  Inc.,  "Improved  Pregnancy  Outcome 
Program  Evaluation  Manual,"  report  No. 
OPEL  77-2,  March  1977. 

6.  Health  Services  Administration/OPEL, 
"PSRO:  An  Evaluation  of  the  Professional 
Standards  Review  Organization  Program," 
vol.  1-11,  report  No.  OPEL  77-12,  October 
1977. 

"For  further  discussion  of  this  subject,  see  ch.  6  of  this  report. 
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7.  Transaction  Systems,  Inc.,  "Credentialling  of 
Health  Care  Professionals:  Its  Costs  and  Some 
Alternatives  for  the  Health  Services  Adminis- 
tration," March  1975. 

8.  General  Research  Corporation,  "PSRO  Pro- 
gram Cost  Study,"  April  1974. 

9.  Geomet,  Inc.,  "Cost  and  Utilization  Evaluation 
of  Migrant  Ambulatory  Care  Entitlement  Pro- 
grams," March  1979. 

10.  Geomet,  Inc.,  "Comparative  Cost  and  Finan- 
cial Analysis  of  Ambulatory  Care  Providers," 
June  1976. 

11.  Health  Services  Administration,  "Migrant  Am- 
bulatory Care  Entitlement  Programs  -  Cost  and 
Utilization  Evaluation  of  Rural  Health  Initia- 
tive," October  1978. 

Health  Care  Financing  Administration 

HCFA  is  responsible  for  administering  the  medi- 
care, medicaid,  Professional  Standards  Review  Orga- 
nization (PSRO),  and  End-Stage  Renal  Disease  Pro- 
grams. It  has  conducted  several  studies  of  the  eco- 
nomic toll  of  illness  and  the  cost  of  health  care  fi- 
nancing. These  include: 

1.  B.  S.  Cooper  and  D.  P.  Rice,  "The  Economic 
Costs  of  Illness  Revisited,"  Social  Security  Bul- 
letin, 39(2),  February  1976. 

2.  D.  P.  Rice,  "Estimating  the  Cost  of  Illness," 
Health  Economics  Series  No.  6,  Public  Health 
Service,  1966. 

3.  R.  M.  Gibson  and  C.  R.  Fisher,  "National 
Health  Expenditures,  Fiscal  Year  1977,"  Social 
Security  Bulletin,  41(7):3,  1978. 

In  addition,  HCFA  is  currently  sponsoring  an  effort 
aimed  at  encouraging  the  integration  of  cost  contain- 
ment information  into  medical  school  curricula: 

1.  American  Association  of  Medical  Schools,  "A 
Primer  on  Quality  Assurance  and  Cost  Con- 
tainment for  Faculty  and  Students." 
Also,  HCFA  has  sponsored  a  study  which  indirectly 
employed  the  cost-effectiveness  concept: 

1.  Massachusetts  Department  of  Public  Welfare, 
"Cost  Effectiveness  Under  Medicaid,"  1978. 
This  study  was  designed  to  investigate  whether  medi- 
cal criteria  could  be  utilized  within  a  medicaid  pro- 
gram to  reduce  inappropriate  and  ineffective  medical 
care.  Problems  in  implementation,  however,  led  to 
HCFA's  termination  of  funding  for  the  study. 

In  addition,  HCFA  has  conducted  an  assessment  of 
the  costs  and  the  benefits  of  PSROs.  Under  Title  XI  of 
the  Social  Security  Act,  PSROs  are  responsible  for 
reviewing  the  medical  necessity,  quality,  and  appro- 
priateness of  federally  financed  health  programs.  In 
order  to  perform  this  function,  PSROs  must  develop 


norms,  criteria,  and  standards  for  the  appropriate 
utilization  and  acceptable  quality  of  health  care  serv- 
ices. A  1979  report  by  HCFA,  elaborating  on  a  previ- 
ous evaluation  (288),  concluded  that  the  savings  pro- 
duced by  PSROs  through  reduced  use  of  hospital 
services  by  medicare  patients  exceed  the  cost  of 
PSRO  review  (285).  Subsequently,  this  finding  was 
used  in  PSRO  program  justification  before  Congress. 

Two  other  Government  evaluations  of  PSROs, 
however,  have  challenged  HCFA's  findings.  One,  a 
study  by  the  Congressional  Budget  Office  found  that 
PSROs  only  slightly  reduce  medicare  patients'  use  of 
hospital  services  and  concluded  that  the  "PSRO  pro- 
gram probably  yields  a  net  loss"  (121).  Similarly,  a 
1979  General  Accounting  Office  study  stated  that  the 
data  HEW  reviewed  in  its  1977  and  1978  evaluations 
"are  not  based  on  appropriate  hospital  statistics"  and 
that  several  estimates  of  cost  savings  attributable  to 
PSROs  were  overstated  (245). 

Office  of  Human  Development  Services 

OHDS  of  DHHS  administers  a  wide  range  of  pro- 
grams which  are  designed  to  aid  children,  youth, 
families,  the  aged,  the  handicapped,  and  native 
Americans.  Over  the  last  few  years,  since  the  first  ap- 
plication of  cost-benefit  framework  to  vocational 
rehabilitation  by  Ronald  Conley  in  1965  (124), 
OHDS  has  conducted  a  number  of  CBAs  of  rehabili- 
tation services.  OHDS,  as  well  as  many  State  voca- 
tional rehabilitation  agencies,  has  used  this  type  of 
analysis  to  justify  increased  Government  spending  on 
training  programs  for  the  disabled. 

The  application  of  CBA  to  vocational  training  ap- 
pears to  have  been  facilitated  by  the  existence  of  both 
a  traditional  set  of  program  goals  (e.g.,  gainful  em- 
ployment of  the  disabled)  and  a  long-term  informa- 
tion system  about  such  factors  as  increased  earnings 
and  reduced  costs  of  special  medical  or  custodial  care 
(479).  Among  the  many  federally  and  State-spon- 
sored studies  of  this  type  are  the  following: 

Federally  Sponsored  Studies 

1.  Rehab  Group,  Inc.,  "Feasibility  Study  for  the 
Evaluation  of  Methodologies  for  Cost  Benefit 
Analysis  of  Restoration  Services  in  Rehabilita- 
tion," prepared  for  OHDS,  October  1978. 

2.  R.  O.  Washington,  D.  Y.  Rowland,  C.  Hairston, 
"Quantification  of  Human  Services  Outcomes:  A 
Manual  for  Applying  Program  Budgeting,  Sys- 
tems Analysis  and  Cost-Benefit  Analyses  to  Hu- 
man Service  Programs,"  prepared  for  Social  and 
Rehabilitation  Services,  August  1974. 
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3.  L.  Mars,  "An  Exploratory  Cost-Benefit  Analysis 
of  Vocational  Rehabilitation,"  prepared  for  the 
Vocational  Rehabilitation  Administration,  1967. 

4.  D.  E.  Carter  and  T.  J.  George,  "A  Manual  for  Re- 
porting and  Analyzing  the  Costs  and  Benefits  of 
Social  Services,"  prepared  for  Social  and  Rehabil- 
itation Services,  November  1973. 

5.  H.  Emlet,  et  al.,  "Estimated  Health  Benefits  and 
Cost  of  Post-Onset  Care  for  Stroke,"  prepared  by 
Analytic  Services,  Inc.,  in  cooperation  with  Johns 
Hopkins  University,  for  Social  and  Rehabilitation 
Services,  September  1973.  [This  is  an  assessment 
of  the  cost  and  benefits  of  poststroke  care  in  the 
population  of  three  States.] 

6.  D.  R.  Matlack,  "Cost  Effectiveness  of  Spinal 
Cord  Injury  Center  Treatment,"  prepared  by  the 
National  Paraplegia  Foundation  for  OHDS,  1974. 
[This  report  provides  a  CEA  of  two  alternatives 
spinal  cord  injury  treatment  methods.] 

7.  Stanford  Research  Institute,  "Feasibility  and  Cost 
Effectiveness  of  Alternate  Long-Term  Care  Set- 
tings," prepared  for  Social  and  Rehabilitation 
Services,  May  1978.  [This  is  a  pilot  study  which  is 
designed  to  identify  methods  that  could  be  used 
to  determine  the  relative  cost-effectiveness  of 
various  alternate  types  of  long-term  care  set- 
tings.] 

State-Sponsored  Studies 

1.  Michigan  Department  of  Education,  Division  of 
Vocational  Rehabilitation,  "A  Benefit-Cost 
Analysis  of  Vocational  Rehabilitation  Programs 
in  the  State  of  Michigan,"  1970. 

2.  R.  D.  Struthers,  "A  Benefit/Cost  Model  Devel- 
oped in  a  State  Vocational  Rehabilitation  Agen- 
cy: Michigan  Vocational  Rehabilitation,"  Jour- 
nal of  Rehabilitation  Administration,  July  1977. 

3.  E.  Wright,  "A  Benefit-Cost  Analysis  of  Voca- 
tional Rehabilitation  Programs  in  the  State  of 
Michigan,"  prepared  for  the  Michigan  Rehabili- 
tation Service,  Lansing,  Mich.,  1970. 

4.  C.  M.  Grigg,  A.  G.  Holtman,  and  P.  F.  Martin, 
"Vocational  Rehabilitation  for  Disabled  Public 
Assistance  Clients:  An  Evaluation  of  Fourteen 
Research  and  Demonstration  Projects,"  pre- 
pared for  the  Institute  for  Social  Research, 
Florida  State  University,  Tallahassee,  Fla.,  1969. 

5.  J.  E.  Muthard,  et  al.,  "The  Vocational  Rehabili- 
tation of  Public  Assistance  Recipients:  A  Na- 
tional Survey,"  prepared  for  the  Rehabilitation 
Research  Institute,  University  of  Florida, 
Gainesville,  Fla.,  1976. 

6.  K.  W.  Reagles  and  G.  N.  Wright,  "A  Benefit- 
Cost  Analysis  of  the  Wood  County  Project:  An 


Illustrated  Lecture,"  prepared  for  the  University 
of  Wisconsin  Regional  Rehabilitation  Institute, 
1971. 

7.  M.  Berkowitz,  "Benefit  Cost  Analyses:  An  Ap- 
plication to  a  Vocational  Rehabilitation  Project: 
An  Interim  Report,"  prepared  for  the  Bureau  of 
Research,  Rutgers  University,  New  Brunswick, 
N.J.,  May  1976. 

8.  C.  Cole  and  R.  Dodson,  "An  Introduction  to 
Cost  Benefit  Analysis  of  the  Vocational  Rehabil- 
itation Program:  A  Model  for  Use  by  State 
Agencies,"  prepared  for  the  Institute  of  Urban 
and  Regional  Development,  University  of  Cali- 
fornia, Berkeley,  Calif.,  October  1972. 

9.  F.  C.  Collignon,  "A  Working  Outline  for  Cost- 
Benefit  Analyses  of  Vocational  Rehabilitation 
Programs,"  prepared  for  the  Institute  of  Urban 
and  Regional  Development,  University  of  Cali- 
fornia, Berkeley,  Calif.,  November  1971. 

10.  D.  W.  Dunlop,   "Benefit/Cost  Analysis:  An 
Analytical  Framework  for  Vocational  Rehabili- 
tation," prepared  for  the  Michigan  Vocational 
Rehabilitation  Service,  Lansing,  Mich.,  1969. 
It  is  generally  conceded  that  these  studies  possess  a 
number  of  serious  limitations.  Several  of  the  anal- 
yses, particularly  those  conducted  by  State  voca- 
tional rehabilitation  agencies,  suffer  from  poor  meth- 
odological design  and  inadequate  data  (52).  In  addi- 
tion, cost  estimates  are  often  imprecise  because  of  the 
difficulty  of  measuring  rehabilitation  costs,  and 
benefit  assessments  tend  to  represent  general  esti- 
mates because  a  number  of  benefits  are  psychic  or  in- 
tangible and  do  not  lend  themselves  to  quantification 
(123). 

The  1973  amendments  (Public  Law  93-112)  to  the 
1965  vocational  rehabilitation  legislation  (Public  Law 
89-333)  mandated  that  Federal  officials  broaden  their 
efforts  and  work  towards  achieving  the  goal  of  inde- 
pendent living  for  the  severely  disabled.  Measure- 
ment of  the  costs  and  benefits  of  services  to  the  se- 
verely handicapped  is  very  difficult,  though,  because 
of  the  problem  of  expressing  in  economic  terms  the 
worth  of  independent  living.  Further,  many  policy- 
makers emphasize  the  importance  of  considering  to- 
gether with  the  costs  of  rehabilitation  programs  their 
humanitarian  goals,  and  they  argue  that  a  traditional 
cost-benefit  framework  is  inappropriate  for  this  pur- 
pose (123,460,599). 

Currently,  OHDS  is  trying  to  deal  with  some  of 
these  problems  and  make  CBAs  more  useful  for  its 
decisionmaking  by  sponsoring  the  development  of  a 
more  sophisticated  cost-benefit  model.  At  the  same 
time,  however,  several  officials  emphasized  the  dan- 
ger of  using  CBA  without  understanding  its  limita- 
tions (479).  Many  seem  to  agree  with  John  Noble's 
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observation  made  in  1977  that  "the  state-of-the-art 
needs  substantial  upgrading  before  CBAs  can  be 
taken  seriously  as  a  guide  to  priority-setting  in  the 
field  of  rehabilitation"  (461). 


Veterans  Administration 

VA  operates  the  largest  centrally  directed  hospital 
and  clinic  system  in  the  United  States.  VA  also  is  ex- 
tensively involved  in  medical  and  health  services  re- 
search. Apart  from  one  current  study  of  the  cost-ef- 
fectiveness of  hospice  care,  however,  VA  has  not 
been  involved  in  conducting  cost-effectiveness  or 
cost-benefit  studies.  Health  systems  research  officials 
intend  to  study  the  results  of  the  hospice  study,  'An 
Evaluation  of  the  Wadsworth  Palliation  Treatment 
Programs,"  which  is  to  be  completed  in  1983,  in 
order  to  assess  the  feasibility  of  using  CEA  (184). 

Officials  within  VA  are  considering  intensifying 
VA's  efforts  in  the  evaluation  of  health  care  tech- 
nology and  are  therefore  interested  in  exploring  the 
cost-benefit  methodology.  VA's  new  health  services 
R&D  director,  Dr.  Richard  J.  Green,  has  expressed  a 
special  interest  in  the  use  of  evaluation  techniques 
such  as  CEA  in  the  examination  of  health  care  issues. 
A  few  of  the  areas  in  which  VA  hopes  to  employ 
CEA  techniques  in  the  future  are  rehabilitation  medi- 
cine, alternative  models  of  care,  extended  care  pro- 
grams, and  contracted  services.  An  area  that  VA 
hopes  to  focus  more  attention  on  in  the  future  is  pre- 
ventive care  and  preventive  care  packages  for  veter- 
ans. It  is  uncertain  at  this  point  whether  CEA  will 
play  a  role  in  the  evaluation  and  planning  in  this 
area,  but  it  seems  clear  that  there  is  great  interest  in 
its  use. 

State  and  Local  Governments  and 
Nongovernmental  Organizations 

Although  a  few  State  and  local  governments  and 
nongovernmental  organizations  have  had  experience 
with  cost-effectiveness  and  cost-benefit  studies,  they 
appear  to  use  such  analyses  only  rarely.  This  is  not 
surprising  for  at  least  two  reasons.  First,  these  groups 
traditionally  devote  far  less  funds  and  staff  to  evalua- 
tion than  the  Federal  Government  does.  And  second, 
State,  local,  or  regional  CEA/CBAs  tend  to  be  expen- 
sive because  the  necessary  data  are  generally  difficult 
to  obtain.  Where  State  and  local  CEA/CBAs  have 
been  conducted,  their  performance  has  usually  re- 
flected individual  staff  interest  in  CEA/CBA  tech- 
niques. Perhaps,  the  one  major  exception  to  this  gen- 
eralization lies  in  the  area  of  rehabilitation,  where 
many  State  and  local  governments  have  followed  the 


Federal  Government's  lead  in  using  cost-benefit 
studies  to  justify  investment  in  vocational  training.15 
As  the  following  lists  of  State  and  local  studies 
show,  apart  from  vocational  rehabilitation  studies, 
most  State  and  local  CBAs  have  been  conducted  in 
Massachusetts  and  New  York  (19). 

State  Government  Studies 

1.  Massachusetts  Department  of  Public  Health, 
"Cost-Benefit  Analysis  of  New  Born  Screening  for 
Metabolic  Disorders,  N.  Eng.  J.  Med.,  291:1414, 
1974. 

2.  Massachusetts  Department  of  Public  Health  (M. 
E.  Farber  and  S.  N.  Finkelstein),  "A  Cost  Benefit 
Analysis  of  A  Mandatory  Premarital  Rubella-An- 
tibody Screening  Program,"  JV.  Eng.  J.  Med., 
Apr.  12, 1979. 

3.  D.  S.  Shepard,  andM.  Thompson,  "The  Econom- 
ics of  Prevention:  The  Method  of  Cost-Effective- 
ness Analysis,"  prepared  for  the  Office  of  State 
Health  Planning,  Boston,  Mass.,  July  1977,  re- 
vised and  published  as  "First  Principles  of  Cost- 
Effectiveness  Analysis  in  Health,"  Public  Health 
Reports,  November/December  1979. 

4.  A.  Rogers  and  P.  M.  Bloomburgh,  "Cost-Effec- 
tiveness Under  Medicaid,"  Dec.  31,  1979,  pre- 
pared by  staff  of  the  Massachusetts  Department 
of  Public  Welfare  for  the  Health  Care  Financing 
Administration.  [HCFA  terminated  this  project 
before  its  scheduled  completion.] 

5.  E.  L.  Hannany  and  J.  K.  Graham,  "A  Cost-Bene- 
fit Study  of  a  Hypertension  and  Screening  and 
Treatment  Program  at  the  Work  Setting,"  In- 
quiry, 4(4): 345,  December  1978,  prepared  for  the 
New  York  State  Department  of  Health,  Bureau  of 
Disease  Control. 

6.  J.  K.  Paperfuss  and  B.  C.  Ejeldshed,  "Cost-Benefit 
Study  of  Selected  Interventions  in  Control  and 
Prevention  of  Tuberculosis  in  the  State  of  Michi- 
gan," prepared  by  the  Michigan  Division  of 
Health  for  the  Center  for  Disease  Control,  De- 
cember 1969. 

Local  Government  Studies 

1.  A.  Leslie,  "A  Benefit/Cost  Analysis  of  New  York 
City  Heroin  Addiction  Problems  and  Programs, 
1971,"  in  Analysis  of  Urban  Health  Problems, 
edited  by  I.  Levinson  and  J.  Weiss,  New  York: 
Spectrum  Publications,  Inc.,  1976. 

2.  M.  L.  Ingbar,  "Data  System  To  Evaluate  Cost-Ef- 
fectiveness of  Ambulatory  Health  Services  to  the 

15See  the  section  of  this  appendix  on  the  Office  of  Human  Development 
Services  (p.  154)  for  a  discussion  of  these  State  and  local  studies. 
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Poor  in  Cambridge,  Massachusetts,"  1971,  pre- 
pared by  staff  of  the  Department  of  Health,  Hos- 
pitals, and  Welfare,  Cambridge,  Mass. 

Nongovernmental  Studies 

Among  nongovernmental  agencies,  cost-effective- 
ness and  cost-benefit  studies  also  have  been  infre- 
quent. Further,  as  the  following  list  shows,  those 
studies  that  have  been  conducted  have  often  been 
funded  by  the  Federal  Government. 

1.  D.  R.  Matlack,  "Cost-Effectiveness  of  Spinal 
Cord  Injury  Center  Treatment,"  prepared  by  the 
National  Paraplegia  Foundation  for  the  Office  of 
Human  Development  Services,  HEW,  1974. 

2.  A.  Zuvekas,  "Cost-Effectiveness  of  Community 
Health  Centers,"  prepared  by  the  National  Asso- 
ciation of  Community  Health  Centers  for  the  Na- 
tional Center  for  Health  Services  Research,  1979. 
[This  study  provides  a  cost-effectiveness  method- 
ology for  evaluation  of  CHCs.] 


3.  M.  M.  Kristein  and  S.  Jonas,  "A  Cost-Effective- 
ness Manual  for  HSA  Planning.  Prepared  by  the 
American  Health  Foundation  for  the  National 
Center  for  Health  Services  Research,  1980.  [This 
study  will  provide  an  HSA  cost-effectiveness 
manual  with  particular  emphasis  on  health  pro- 
motion areas.] 

4.  M.  M.  Kristein  and  C.B.  Arnold,  "Mammograph- 
ic  Screening  for  Breast  Cancer:  The  Economic 
Analysis,"  paper  presented  at  the  American  Pub- 
lic Health  Association,  Oct.  17,  1978. 

5.  M.  M.  Kristein,  "Cost-Effectiveness  of  Various 
Smoking  Cessation  Methods,"  prepared  by  the 
American  Health  Foundation,  1978. 

6.  M.  M.  Kristein,  "The  Economics  of  Secondary 
Prevention:  Screening  for  Disease:  An  Example 
From  Colo-Rectal  Cancer,"  prepared  by  the 
American  Health  Foundation  for  the  National 
Cancer  Institute,  1978. 
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Introduction 

Formal  CEA  and  CBA  are  applied  to  a  wide  range 
of  topics  in  a  number  of  diverse  areas.  A  CEA  or 
CBA  can  be  performed  by  a  single  analyst  or  by  a 
dozen  or  more.  An  analysis  can  take  a  few  months  to 
complete  or  may  require  more  than  a  year.  The  size 
of  the  problem,  the  availability  and  quality  of  data, 
the  complexity  of  the  issues  involved,  and  the  pres- 
ence of  the  right  mix  of  professionals  combine  to  ex- 
ert considerable  influence  on  the  cost,  the  quality,  the 
usefulness,  and  the  credibility  of  the  analysis. 

A  major  focus  of  The  Implications  of  Cost-Effec- 
tiveness Analysis  of  Medical  Technology  project  was 
on  the  feasibility  and  implications  of  using  CEAs 
and/or  CBAs  in  the  health  care  decisionmaking  proc- 
ess. An  important  component  of  that  focus  are  the  di- 
rect resource  costs  of  performing  these  types  of  anal- 
yses. The  cost  of  producing  a  CEA  or  CBA  will  sig- 
nificantly influence  the  use  of  the  methodologies. 
What  are  the  resource  needs,  the  problems  likely  to 
be  encountered,  and  the  time  needed  to  produce 
quality  analyses?  What  variables  influence  the  re- 
sources used  or  needed  to  perform  various  CEAs  or 
CBAs  in  the  health  care  system?  The  feasibility  and 
potential  impacts  of  using  CEA  or  CBA  in  the  health 
care  system  are  directly  tied  to  such  questions.  To 
identify  and  discuss  the  range  of  answers,  OTA  con- 
ducted a  survey  of  the  resource  costs  of  CEA/CBA. 

Method 

The  survey  instrument  OTA  used  was  a  question- 
naire designed  to  explore  the  types  and  amounts  of 
resources  required  to  perform  a  CEA  and/ or  CBA  of 
various  health  care  technologies.  (See  the  addendum 
to  this  appendix  for  a  copy  of  the  questionnaire.) 
Two  types  of  information  were  desired:  first,  the 
resource  costs  of  performing  actual  CEAs  or  CBAs 
that  have  been  published  in  the  literature;  and  sec- 
ond, the  resources  the  sample  group,  comprised  of 
the  individuals  who  performed  those  analyses,  felt 
would  be  required  to  staff  a  hypothetical  research 
team  responsible  for  conducting  CEAs  or  CBAs  on 
medical  technologies  on  a  regular  and  continuing  ba- 
sis. In  essence,  the  survey  sought  a  listing  of  re- 
sources that  had  been  used  for  actual  studies  and  esti- 
mates of  resources  needed  to  perform  CEAs  and 
CBAs  on  a  routine  basis. 

The  sample  population  was  chosen  on  the  basis  of 
an  analysis  of  the  health  care  literature  and  discus- 
sions with  analysts  in  the  health  care  area.  Thirty- 


five  studies  were  selected  by  this  process;  no  attempt 
was  made  to  randomize  the  selection  process.  The 
survey  was  sent  to  analysts  identified  as  having  done 
"quality"  work  or  whose  studies  were  cited  frequent- 
ly in  the  health  care  literature.  Not  necessarily  all 
quality  studies  were  identified  and  selected  by  this 
process. 

Twenty-two  responses  were  returned.  Eighteen  re- 
spondents answered  the  questions  pertaining  to  the 
resources  actually  used  in  performing  the  published 
analyses.  Twenty-two  respondents  answered  the 
questions  related  to  the  resource  needs  of  the  hypo- 
thetical CEA/CBA  team  described  in  the  survey. 

Results 

Resources  Used  in  Published  CEAs  and  CBAs 

The  number  of  professional-level  people  directly 
involved  in  a  single  analysis  ranged  from  1  to  10  per 
study  team,  with  the  mode  at  3  and  the  mean  at  3.7. 
There  was  insufficient  information  to  determine  the 
degree  of  effort,  or  percentage  of  time,  that  the  vari- 
ous professionals  devoted  to  the  studies.  The  re- 
sponses were  so  varied  and  wide  ranging  that  it  was 
difficult  to  characterize  the  amount  of  time  that,  say, 
a  physician  or  economist  spent  on  a  given  analysis. 
The  amount  of  full-time  effort  devoted  to  the  studies 
ranged  from  as  little  as  a  single  day  to  as  much  as  a 
full  year. 

One  trend  that  did  emerge  from  the  survey  was  the 
use  of  physicians  on  the  study  teams.  Only  one  study 
did  not  have  a  physician  directly  involved  in  the 
analysis;  the  remaining  studies  had  at  least  one  physi- 
cian, and  several  had  two  or  more  physicians,  as  part 
of  their  research  group.  The  professions  that  were 
used  in  performing  actual  CEAs  or  CBAs  on  health 
care  topics  are  summarized  in  table  C-l. 

The  survey  also  attempted  to  identify  the  informa- 
tion used  by  the  analysts.  Their  responses  are  sum- 
marized in  table  C-2.  The  types  of  data  that  the  vari- 
ous analysts  used  tended  to  be  study-specific.  Re- 
sponses included  data  on  investment  and  operating 
costs,  health  education  program  costs,  drug  costs, 
screening  costs,  travel  time  to  hospital,  ambulance 
response  time,  physician  fees,  and  on  a  host  of  other 
cost  and  benefit  variables  that  cut  across  the  four  ma- 
jor categories  listed  in  table  C-2. 

The  final  topic  that  the  respondents  were  asked  to 
address  related  to  the  data  sources  they  used  and  the 
problems  they  had  in  obtaining  their  information. 
Their  responses  are  summarized  in  table  C-3. 
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Table  C-1.— Professions  Used  in  Performing 
Health-Related  CEA/CBAs 


Personnel  used 

Frequency  of  response 

1 7 
I  1 

7 

b 

Hospital  administration  

3 

Systems  analysis  

0 

A 

4 

A 

4 

Computer  analysis  

3 

Computer  programing  

2 

Public  administration  

2 

Business  administration  

2 

Medical  student  

2 

Accountant   

1 

1 

Psychology  

1 

SOURCE:  Office  of  Technology  Assessment. 

Table  C-2.— Types  of  Data  Used  in 

Health-Related  CEA/CBAs 

Data  used 

Frequency  of  response 

Morbidity  

17 

Mortality  

16 

Epidemiology  

13 

Health  services  utilization  

8 

SOURCE:  Office  of  Technology  Assessment. 

Table  C-3. — Availability  and  Sources  of  Data 

for  Health-Related  CEA/CBAs 

Data  characteristics 

Frequency  of  response 

Availability  of  data 

Readily  available  

11 

Difficult  to  obtain  

15 

Almost  impossible  to  obtain  

6 

Had  to  purchase  data  

0 

Did  original  research  

8 

Collected  from  existing  sources  . . 

18 

Data  was  free  

18 

Data  was  inexpensive  

7 

Data  was  expensive  

4 

Sources  of  data 

Data  obtained  from  public  sources 

10 

Data  obtained  from  journals  

16 

Data  obtained  from  books  

8 

Data  obtained  from  Government 

reports,  etc  

16 

Data  obtained  from  private 

sources  (industry,  insurance 

companies,  etc.)  

9 

Data  obtained  from  nonprofit 

organization  

7 

Data  obtained  from  other  sources 

State  government  

1 

Expert  opinion  

2 

Unpublished  reports  

1 

Clinical  trial  information  

2 

SOURCE:  Office  of  Technology  Assessment. 


Several  respondents  noted  that,  although  the  data 
may  have  been  free,  the  time  and  effort  required  to 
obtain  it  was  expensive.  Another  problem  seemed  to 
be  determining  whether  the  data  existed,  and  if  so, 
who  had  it. 

Resource  Needs  of  an  "Ideal" 
CEA/CBA  Research  Team 

The  professions  most  frequently  cited  as  being  es- 
sential components  of  an  office  designed  to  perform 
CEAs  and  CBAs  of  health-care-related  technologies 
on  a  regular  and  continuing  basis  are  shown  in  table 
C-4. 

Table  C-4.— Professions  and  Support  Services  for 
an  "Ideal"  CEA/CBA  Research  Team 


Professions/support  services  cited  Frequency  of  response 


Profession  cited 

Economics   21 

Medicine   17 

Statistics   14 

Quantitative  analysis3   13 

Computer  analysis   9 

Epidemiology   9 

Public  health/public  policy   7 

Engineering   6 

Psychology   4 

Hospital  administration   2 

Medical  sociology   2 

Sociology/behavioral  science   2 

Medical  student   1 

Support  services 

Secretary   16 

Research  assistant   15 

Computer  programer   9 

Administrator   8 

Librarian   3 

Accountant   2 


aSystems  analysis,  operations  research,  and  decision  analysis. 
SOURCE:  Office  of  Technology  Assessment. 

Several  respondents  specifically  noted  that  provi- 
sions must  be  made  for  hiring  the  necessary  consult- 
ants or  experts  as  needed  for  assistance.  Many  re- 
spondents also  cited  the  need  for  more  than  one  of 
each  professional  (e.g.,  two  physicians,  three  econo- 
mists, several  secretaries,  six  research  assistants,  etc.) 
to  staff  the  research  group. 

The  number  of  studies  that  this  "ideal"  office 
would  be  able  to  perform  was  estimated  to  be  6  to  10 
studies  per  year  with  8  to  12  full-time  professionals 
and  the  necessary  consultants,  research  assistants, 
and  support  staff  present.  The  estimated  cost  of  the 
research  group  was  between  $400,000  and  $500,000 
per  year. 

The  estimated  cost  per  study,  not  necessarily  re- 
lated to  the  aforementioned  total  office  cost  esti- 
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mates,  covered  a  broad  range.  The  graph  in  figure 
C-l  summarizes  and  illustrates  the  wide  range  and 
distribution  of  the  cost-per-study  estimates. 

Many  of  the  cost  estimates  were  accompanied  by 
reservations  indicating  a  close  correlation  between 
the  cost  of  the  study  and  the  extensiveness  and  severi- 
ty of  problems  with  information  needs,  the  stage  of 
the  technology's  development,  the  complexity  of  the 
problem  being  addressed,  computer  needs,  and  the 


costs  to  develop  the  study  model.  Though  not  the 
only  variables  cited,  these  problems  received  fre- 
quent mention  by  the  respondents. 

Discussion 

The  purpose  of  this  survey  was  to  identify  infor- 
mation and  personnel  variables  that  might  affect  the 
"cost"  of  a  CEA/CBA,  solicit  opinions  regarding  the 


Figure  C-1.— Range  of  Estimated  Costs  per  CEA/CBA  Study 


Cost-per-study  estimates3 
(in  thousands  of  dollars) 


$600r 


500  - 


200  - 


Individual  responses 

■  =  High  estimate 
•  =  Low  estimate 

aMost  of  the  respondents  included  a  high  and  low  estimate  of  the  cost-per-study. 
SOURCE:  Office  of  Technology  Assessment. 
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resources  required  to  do  CEA  or  CBAs  on  a  routine 
basis,  and  to  develop  a  range  of  estimates  of  the  re- 
sources used  to  do  published  CEAs  or  CBAs.  To  a 
large  degree,  these  goals  were  realized.  Because  of  the 
survey's  design,  however,  no  firm  conclusions  are 
possible. 

The  scope  of  the  survey  was  limited  in  the  sense 
that  the  process  used  to  select  the  sample  population 
was  restricted  and  arbitrary  since  the  criteria  used  to 
identify  the  "quality"  CEAs  and  CBAs  were  highly 
subjective.  The  survey  sample  was  limited  even  fur- 
ther because  its  focus  was  restricted  to  CEAs  and 
CBAs  done  only  on  health  care  topics.  The  survey 
sample  may  have  been  further  distorted  by  the  char- 
acteristics of  the  types  of  issues  selected  by  research- 
ers. Possibly,  the  CEA  and  CBA  studies  done  to  date 
have  been  "easy"  ones.  Analyses  of  health  care  topics 
may  have  been  directed  at  areas  that  have  had  the 
most  or  best  data  available  or  that  were  the  most 
highly  visible  in  the  public  eye.  On  the  other  hand,  a 
counterargument  could  be  made.  Since  the  field  is 
relatively  new,  perhaps  more  work  was  required  to 
develop  the  methods,  more  analysts  were  needed  to 
perform  quality  work,  and  in  essence,  added  effort 
was  needed  to  establish  the  groundwork  that  other 
analysts  can  use.  Whichever  the  case,  it  is  difficult  to 
know  what  effects  these  variables  will  have  on  the 
sample  results. 

The  survey  was  broadly  focused  in  that  no  attempt 
was  made  to  differentiate  between  the  various  types 
of  analyses.  For  example,  no  attempt  was  made  to 
group  the  studies  according  to  their  technological 
focus  (i.e.,  diagnosis  v.  therapy,  procedure  v.  drug, 
or  systems-based  technology).  Likewise,  no  attempt 
was  made  to  determine  if  some  analyses  were  consid- 
ered more  complex  or  sophisticated,  or  if  some  had 
more  "value"  to  the  research  community  or  to  the 
policy  process,  than  others.  Although  these  variables 
are  very  important,  it  was  beyond  the  scope  of  this 
survey  exercise  to  investigate  them  to  the  degree  nec- 
essary to  form  estimates  or  conclusions. 

At  best,  the  survey  results  suggest  a  lack  of  consen- 
sus on  most  aspects  of  funding  or  staffing  resources 
required  for  CEAs  or  CBAs.  A  good  example  of  this 
lack  of  agreement  are  the  estimates  of  what  it  might 
cost  to  perform  a  single  CEA  or  CBA.  In  large  part, 
however,  the  many  reservations  that  the  respondents 
included  regarding  the  effect  of  data  problems  and 
the  complexity  of  the  issue(s)  being  examined  on  the 
cost  of  the  study  would  explain  the  wide  range  of 
estimates  provided.  Although  there  were  few  sur- 
prises regarding  the  types  of  professionals  or  support 
staff  required  to  do  CEAs  or  CBAs,  there  was  an  in- 


teresting change  in  views  in  the  responses  received 
for  the  questions  related  to  the  types  of  professionals 
actually  used  to  perform  specific  analyses  and  those 
related  to  the  types  of  professionals  recommended  to 
staff  the  hypothetical  research  team.  All  but  one  ac- 
tual study  included  a  physician  as  part  of  the  research 
group,  but  only  seven  included  an  economist.  The 
hypothetical  research  teams,  however,  leaned  more 
heavily  toward  the  inclusion  of  economists.  This  ap- 
parent shift  toward  economists  may  not  indicate  any 
real  change  in  the  perceived  need  for  physicians, 
though,  because  several  respondents  indicated  a  need 
for  physicians  and  scientists  to  serve  as  consultants  to 
the  hypothetical  research  group.  Thus,  some  of  the 
respondents  to  the  survey  listed  physicians  in  another 
category.  The  shift  to  more  economists,  however, 
was  not  explained. 

Data  needs  and  problems  appeared  to  be  a  signifi- 
cant factor  for  all  the  respondents.  The  large  range  of 
cost-per-study  estimates  was  directly  tied  to  the 
availability  and  quality  of  data.  Information  needs 
were  cited  much  more  often  than  factors  such  as 
complexity  of  the  problem  being  studied  and  stage  of 
development  of  the  technology  as  variables  that  will 
affect  the  cost  of  a  given  study.  To  date,  however, 
the  respondents  seem  to  feel  that  many  of  the  data 
range  from  being  easy  to  difficult,  but  not  impossi- 
ble, to  obtain.  The  data  also  were  very  inexpensive  to 
obtain  and  were  available  from  public  sources. 

The  results  of  OTA's  survey  may  simply  provide  a 
look  at  the  types  of  resources  analysts  have  used  to 
perform  health-care-related  CEAs  and  CBAs  and  at 
the  types  of  resources  they  feel  are  necessary  to  per- 
form them  on  a  routine  basis.  It  is  interesting  to  note 
the  differences  between  the  resources  used  and  the 
resources  that  the  respondents  felt  were  required  to 
do  CEA  and  CBA  on  a  regular  basis.  Several  profes- 
sions that  were  not  used  a  great  deal  by  the  actual 
study  teams  in  the  past  show  up  quite  frequently  on 
the  "ideal"  research  team  list.  Disciplines  such  as 
economics,  statistics,  engineering,  computer  anal- 
ysis, and  computer  programing  were  not  frequently 
used  in  the  actual  studies,  yet  were  cited  several  times 
by  the  respondents  as  being  needed  for  the  hypotheti- 
cal research  group.  As  the  health  care  issues  become 
more  complex  and  as  information  needs  become 
more  demanding,  the  range  of  expertise  needed  to  do 
analyses  will  broaden,  and  the  costs  of  performing 
CEAs  and  CBAs  will  increase.  These  resource  costs 
may  become  an  important  factor  to  be  considered  by 
governments  and  other  institutions  that  must  decide 
whether  or  not  to  do  CEAs  or  CBAs  or  how  many 
they  can  do  in  a  given  time  period. 
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ADDENDUM 
QUESTIONNAIRE  USED  IN  THE 
SURVEY  OF  THE  COSTS  OF  CEA/CBA 

PART  1 

What  is  the  general  cost  range  that  you  feel  is  adequate  to  perform  most 
cost-benefit  or  cost-effectiveness  studies  on  medical  technologies,  techniques, 
procedures  or  systems? 


We  realize  that  the  costs  will  vary  according  to  different  factors  (e.g.,  type  of 
technology,  stage  of  development).    What  are  the  factors  that  must  be  considered 
and  that  are  most  important  in  effecting  these  cost  differences,  and  how  would 
they  affect  the  cost  of  analysis? 


Additional  comments: 


Note:    This  aspect  of  the  survey  (Part  1)  is  very  important  to  us;  therefore  we 
request  that  it  receive  priority  should  you  find  yourself  short  of  time; 
however,  we  do  urge  you  to  complete  all  parts  of  this  survey. 

A.      Hypothetical  situation:    You  are  asked  to  staff  an  office  that  would  be 

responsible  for  carrying  out  cost-effectiveness  or  cost-benefit  studies  of 
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a  range  of  medical-related  technologies,  techniques,  procedures,  and 
systems.    What  are  the  necessary  disciplines  to  carry  out  this  function? 
How  many  of  each? 

1.  6. 


2.    7. 

3.    8. 

4.    9. 

5.  10. 


What  support  services  or  personnel  are  needed?    How  many  of  each? 
1.  M. 


2.  5. 


3.  6. 


PART  2 

The  intent  of  this  part  of  the  survey  (Part  2)  is  to  obtain  a  range  of 
estimates  of  resources  that  have  been  used  to  perform  cost-effectiveness  or 
cost-benefit  studies.    Since  you  have  either  performed  or  directed  such  a  study, 
we  ask  for  your  help  in  providing  us  this  information.    At  the  bottom  of  the 
page,  we  have  indicated  the  study  or  studies  that  we  wish  you  to  address. 

Perhaps  the  easiest  and  quickest  way  for  you  to  help  us  would  be  if  you  just 
send  us  the  budget  breakdown  that  was  developed  for  the  study  or  studies  listed 
for  contract  or  grant  purposes.    Should  you  wish  to  keep  certain  parts  of  it 
confidential,  please  black  them  out. 

If  you  do  not  have  a  budget  breakdown  or  would  prefer  not  to  send  it,  we  ask 
that  you  fill  out  sections  I  and  II  below  instead.    For  those  of  you  who  are 
sending  the  budget  page,  we  ask  that  you  also  fill  out  section  II  only. 
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I.    How  many  professional  level  people  directly  contributed  to  the  analysis? 


A.       What  types  of  training  or  educational  backgrounds  did  these  people 

have  (what  did  they  consider  their  professional  niche)?    Please  check 
off  the  disciplines  involved;  if  more  than  one  person  in  each 
discipline,  please  indicate  how  many.    Please  indicate  level  of 
education,  B.S. ,  M.S.,  Ph.D.,  etc. 

1.  Economics    (Specialty   ) 

2.  Health  care  professional ( s) : 

a.  Physician(s)           practicing  or  academic   

b.  Public  health  _ 

c.  Hospital  administration 

d.  Epidemiologist 

e.  Scientist(s) 

What  discipline(s)   


3.  Systems  analysis 

4.  Operations  research  _ 

5.  Computer  analysis 

a.    computer  programming 

6.  Public  administration   

7.  Lawyer  _ 

8.  Other   


B.      What  level  of  effort  did  these  people  contribute  to  the  study  in  terms 
of  full-time  equivalents  or  person  days,  weeks,  or  months  devoted  to 
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the  analysis?  (State  type  of  professional  and  time  spent.) 
Professional  Time  spent  on  study 

1.     

2.     

3.     

4.     

5.     

6.     

II.    The  information  needs  of  the  analysis: 

A.  What  types  of  information  did  you  need  for  your  analysis?  Please  try 
to  be  fairly  specific  (for  instance,  equipment  cost,  personnel  costs, 
drug  costs,  cervical  cancer  deaths  per  year,  disability  data,  genetic 
disease  prevalence  data,  etc.). 

a.      Direct  cost(s)  data   


Direct  benefit(s)  data 


c.      Indirect  cost  data 


d.      Indirect  benefit  data 


e.      Morbidity  data 
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f .  Mortality  data   

g.  Epidemiological  data   

h.  Health  services  utilization  data   

i.  Other  data  needs   

B.  Was  the  data  required  for  your  study: 

a.  Readily  available   

b.  Difficult  to  obtain 

c.  Almost  impossible  to  get  in  a  usable  form   

C.  In  order  to  obtain  data  in  a  usable  form  for  your  purposes,  did  you 
have  to: 

a.  Purchase  it  (buy  data  tapes,  compiled  lists,  etc.) 

b.  Do  original  research   

c.  Collect  from  existing  sources   

D.  Was  the  data 

a.  Free 

b.  Inexpensive  to  obtain   

c.  Expensive  to  obtain  _  

E.  Was  much  or  all  of  the  data  obtained  from: 

a.    Public  sources  _ 

1.  journals   

2.  books   

3.  government  studies  or  reports  

4.  other 
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b.    Quasi-public  sources,  non-profit  organizations,  foundations,  etc. 


c.    Private  sources  (industry,  insurance  firms,  etc.) 
F.    Additional  comments  or  explanations: 


Appendix  D.— Values,  Ethics,  and  CBA  in  Health  Care 


by  The  Hastings  Center 
Institute  of  Society,  Ethics  and  the  Life  Sciences 

Hastings-on-Hudson,  N.Y. 


Introduction 

The  past  two  decades  have  been  as  hard  on  science 
as  they  have  been  on  many  other  once  venerated  in- 
stitutions, and  for  at  least  one  of  the  same  reasons: 
The  idea  that  it  consists  exclusively  of  delineation  of 
truths  about  the  state  of  the  world  has  been  success- 
fully challenged. 

In  some  instances,  scientific  "facts"  have  been 
revealed  to  be  deliberately  falsified.  Apart  from  tar- 
nishing science's  image,  falsification  has  other  conse- 
quences. If  a  cancer  researcher  fakes  results,  others 
may  spend  months  or  years  and  large  amounts  of 
public  money  attempting  to  replicate  them.  If  a  psy- 
chologist fabricates  IQ  studies  of  twins,  the  data  may 
be  used  in  other  similar  studies  and  reviews,  and  the 
cumulated  erroneous  data  may  be  influential  in  for- 
mulating a  worthless  (at  best),  or  even  a  repressive 
and  openly  racist  (at  worst),  social  policy. 

It  is  one  of  the  axioms  of  the  scientific  enterprise 
that  the  norms  and  internal  controls  of  science  guar- 
antee that  such  abuses  will  ultimately  be  exposed. 
Even  if  that  is  true  for  flagrant  violations,  however,  it 
is  not  clear  that  it  is  true  or  even  can  be  true  for  the 
much  more  subtle  distortions  inherent  in  the  daily 
practice  of  science.  These  generally  involve  nothing 
so  egregious  as  deliberate  falsification  of  data,  but 
rather  much  less  obvious  effects  of  the  researchers' 
biases  and  value  commitments,  many  of  which  are 
unconscious. 

In  general,  the  argument  that  science's  value- 
freedom  is  illusory  has  fallen  less  harshly  on  the 
"hard"  or  natural  sciences  than  on  the  "soft"  or  social 
sciences.  Economics,  however,  is  almost  in  a  class  by 
itself.  Since  its  methods  are  quantitative  almost  by 
definition,  economics  appears  to  be  relatively  "hard" 
compared  to  many  of  the  other  social  sciences.  Partly 
because  of  this,  economics  has  been  one  of  the  least 
criticized  social  sciences. 


NOTE:  This  appendix  includes  nearly  all  of  a  report  prepared 
by  The  Hastings  Center  under  contract  to  OTA.  As  with  all  re- 
ports published  by  OTA,  its  inclusion  does  not  imply  endorsement 
by  the  members  of  the  Technology  Assessment  Board  or  by  any 
advisory  panels.  Although  the  report  focuses  on  CBA,  the  discus- 
sions apply  in  many  instances  to  CEA  as  well.  The  report  should 
not  be  considered  an  exhaustive  treatment  of  ethical  issues  in- 
volved in  the  conduct  and  use  of  CEA/CBA. 


In  recent  years,  that  situation  has  begun  to  change. 
Largely  as  a  result  of  the  "dismal  science's"  dismal 
record  of  forecasting,  economics  has  been  subjected 
to  increasing  criticism.  That  record  has  been  increas- 
ingly frustrating  for  citizens  and  policymakers,  both 
of  whom  must  somehow  cope  with  continuing  uncer- 
tainties about  inflation,  unemployment,  and  other 
economic  indicators.  Deciding  between  big  bang  and 
steady  state  theories  of  the  universe  may  ultimately 
have  profound  effects  on  our  view  of  mankind,  but 
most  people  are  more  concerned  and  more  immedi- 
ately affected  by  theories  of  economics.  The  impact 
of  economics  on  government  policy  and  daily  life 
underscores  the  importance  of  accurate  predictions 
and  conclusions. 

Among  the  currently  fashionable  methods  of  eco- 
nomics are  CEA  and  CBA.  A  major  text  in  the  field 
of  CBA  was  written  by  the  economist  E.  J.  Mishan, 
who  believes  economists  doing  CBA  are  really  asking 
a  question  similar  to  that  asked  by  a  company  ac- 
countant (410).  Instead  of  asking  whether  the  com- 
pany will  be  better  off  engaging  in  one  activity  as  op- 
posed to  another,  the  economist  asks  whether  society 
would  be  better  off  by  undertaking  the  project  under 
study.  In  Mishan's  view,  CBA  applies  standard  eco- 
nomics to  help  advance  the  social  welfare.  The  ra- 
tionale is  based  on  the  Pareto  improvement,1  which 
occurs  if  some  economic  arrangement  makes  people 
better  off  without  making  anyone  worse  off. 

According  to  Herbert  Klarman,  a  pioneer  in  using 
CBA  in  the  health  field,  its  use  by  experts,  and  the 
idea  that  it  should  influence  public  decisions,  is  only 
a  generation  old  (317).  There  are  few  references  to  it 
in  the  literature  before  1958.  More  recently,  CBA  has 
become  a  favorite  tool  among  policymakers  and  has 
been  applied  in  many  areas  of  public  life.  Along  with 
systems  analysis,  CBA  has  been  widely  used  by  the 
military  and  for  major  public  works  projects.2 

Noting  that  investment  in  human  capital  was  a 
popular  theme  in  postwar  economics,  Mishan  sug- 
gests that  it  was  an  easy  step  from  investment  in 
human  productivity  to  investment  in  health  (410). 


'Named  for  the  Italian  economist  and  sociologist  Vilfredo  Pareto 
(1848-1923).  "An  equilibrim  is  said  to  be  'Pareto-optimal'  if  (and  only  if) 
there  is  no  possible  movement  from  it  that  could  make  everyone  better  off" 
(531). 

2One  famous  and  very  controversial  British  study,  for  example,  used  it  to 
help  find  a  site  for  a  third  London  airport. 
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The  rationale  is  simple:  Disease  and  premature  death 
are  costly  to  an  economy,  so  investment  in  medical 
care  can  bring  significant  economic  rewards  by  in- 
creasing human  capital. 

An  increasing  number  of  cost-benefit  and  cost- 
effectiveness  studies  have  been  done  in  the  health 
field  in  the  last  decade,  although  how  influential  such 
studies  have  been  in  the  forging  of  policy  is  not  clear. 
Rashi  Fein  has  pointed  out  that  the  attractiveness  of 
economic  arguments  derives  from  a  belief  that  eco- 
nomics is  value  free  and  objective  (194).  That  makes 
it  seem  particularly  appropriate  for  fields  dealing 
with  social  policy,  such  as  health  care,  which  are 
fraught  with  value  conflicts  and  otherwise  appear  to 
lack  rigorous  guidelines  for  decisionmaking. 

Sociologist  Duncan  MacRae  suggests  that  the  his- 
torical development  of  economics  has  narrowed  the 
discipline's  outlook,  making  it  highly  specialized  and 
scientific  (394).  Whereas  economists  like  Adam 
Smith  all  made  contributions  to  disciplines  other 
than  their  own,  today's  economists  are  either  in- 
terested exclusively  in  economics,  or  they  are  in- 
terested in  extending  economic  theory  to  other  fields 
like  political  science  or  sociology. 

Thus,  MacRae  argues,  welfare  economics  is  a 
much  more  closed  system  than  its  predecessors,  more 
impervious  to  values  that  come  out  of  other  aspects 
of  its  practitioners'  lives.  According  to  R.  M.  Parish, 
this  kind  of  narrow  specialization  is  exactly  what 
CBA  needs  least  (475).  Good  CBA,  he  says,  "seems 
to  call  for  a  combination  of  subtle  theoretical  analy- 
sis, imagination,  and  a  lot  of  hard,  pedestrian  slug- 
ging— the  capacities  to  do  which  are  seldom  com- 
bined in  one  individual." 

The  need  for  economists  with  diverse  capacities 
may  be  especially  great  in  the  area  of  health.  As 
noted  by  R.  N.  Grosse,  there  has  been  little  analysis 
of  the  relationship  between  expenditures  for  and  out- 
comes of  social  programs  (265).  At  the  very  simplest 
level,  we  really  do  not  know  how  medical  services  af- 
fect our  health,  in  part  because  "health"  is  an  abstract 
and  fuzzy  concept.  Health  programs  often  really  can- 
not be  evaluated.  We  lack  agreement  on  their  objec- 
tives and  on  how  to  achieve  them.  We  find  it  hard  to 
estimate  measures  even  when  we  agree  on  them.  We 
cannot  sort  out  cause  and  effect  relationships  because 
there  are  many  inputs  and  outcomes.  Finally,  we 
have  very  little  information  about  programs  whose 
effects  occur  over  a  long  time. 

To  pick  a  program  goal  and  then  attempt  to  eval- 
uate how  well  the  program  is  achieving  that  goal  re- 
quires normative  assessments,  evaluations,  and  as- 
sumptions. CBA,  however,  is  simply  descriptive. 
Norms,  cutoff  points,  decisions  about  where  to  draw 
the  line  between  good  and  bad  outcomes,  all  are  in 


the  mind  of  the  policymaker  and  need  to  be  imposed 
on  the  data  from  outside.  Norms  are  also  in  the  ana- 
lyst's mind,  although  analysts'  commitment  to  the 
disciplinary  ethos  of  quantification  and  objectivity 
may  make  them  genuinely  and  sincerely  unaware 
that  norms  are  an  inevitable  part  of  their  description. 

In  a  book-length  critique  of  CBA,  Peter  Self  asserts 
that  CBA  depends  on  value  judgments  more  than 
most  types  of  information  (556).  Further,  some  of  the 
major  figures  in  economics,  including  many  of  that 
field's  most  thoughtful  practitioners,  share  sub- 
stantial agreement  that  economics  in  general  and 
CBA  in  particular  are  anything  but  norm  and  value 
free. 

This  paper  lays  out  major  values  underlying  the 
quantitative  techniques  of  cost-benefit  and  cost-effec- 
tiveness analysis  in  the  health  field.  It  shows  how 
those  values  can  result  in  ethical  problems,  and 
makes  some  suggestions  for  avoiding  the  difficulties 
the  values  engender.  Though  based  on  extensive 
reading  of  both  the  theoretical  and  technical  litera- 
ture in  this  field,  the  discussion  is  not  exhaustive. 
Our  brief  for  this  exercise  has  been  well  stated  by 
Fein,  who  observed  that  it  is  incorrect  to  suppose  that 
measurement  is  neutral.  "Cost-benefit  analysis,"  he 
says,  "is  too  important  to  be  left  to  analysts  or 
economists"  (194). 

Working  Definitions  of  Cost-Benefit  and 
Cost-Effectiveness  Analysis 

Definitions  of  CEA  and  CBA  in  the  literature  vary. 
OTA  has  defined  them  as  follows: 

Cost-Effectiveness  Analysis  (CEA):  An  analytical 
technique  that  compares  the  costs  of  a  project  or  of 
alternative  projects  to  the  resultant  benefits,  with 
costs  and  benefits /effects  not  expressed  by  the  same 
measure.  Costs  are  usually  expressed  in  dollars,  but 
benefits/effects  are  ordinarily  expressed  in  terms 
such  as  "lives  saved,"  "disability  avoided,"  or  "qual- 
ity-adjusted life  years  (QALYs)  saved,"  or  any  other 
relevant  objectives.  Also,  when  benefits /effects  are 
difficult  to  express  in  a  common  metric,  they  may  be 
presented  as  an  "array." 

Cost-Benefit  Analysis  (CBA):  An  analytical  tech- 
nique that  compares  the  costs  of  a  project  or  techno- 
logical applications  to  the  resultant  benefits,  with 
both  costs  and  benefits  expressed  by  the  same  meas- 
ure. This  measure  is  nearly  always  monetary. 
E.  J.  Mishan  says  (410): 

The  general  question  that  a  cost-benefit  analysis 
sets  out  to  answer  is  whether  a  number  of  investment 
projects,  A,  B,  C,  etc.,  should  be  undertaken  and,  if 
investible  funds  are  limited,  which  one,  two,  or  more, 
among  these  specific  projects  that  would  otherwise 
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qualify  for  admission,  should  be  selected  ...  in  cost- 
benefit  analysis  we  are  concerned  with  the  economy  as 
a  whole,  with  the  welfare  of  a  defined  society,  and  not 
any  smaller  part  of  it. 

Mishan  does  not  concern  himself  specifically  with 

CEA,  but  his  characterization  of  CBA  is  broad 

enough  to  encompass  it. 
For  Prest  and  Turvey  (487): 

Cost-benefit  analysis  is  a  way  of  setting  out  the  fac- 
tors which  need  to  be  taken  into  account  in  making 
certain  economic  choices.  Most  of  the  choices  to 
which  it  has  been  applied  involve  investment  projects 
and  decisions — whether  or  not  a  particular  project  is 
worthwhile,  which  is  the  best  of  several  alternative 
projects,  or  when  to  undertake  a  particular  project. 

This  definition,  too,  is  broad  enough  to  encompass 

both  CEA  and  CBA. 

Not  surprisingly,  therefore,  Weinstein  and  Stason 

observe  (627): 

Cost-effectiveness  analysis  and  benefit-cost  (or 
cost-benefit)  analysis  are  two  related,  but  quite  differ- 
ent, approaches  to  the  assessment  of  health  practices. 
Confusion  frequently  exists  between  the  two  ap- 
proaches, and  many  analyses  that  are  technically  cost- 
effectiveness  analyses  are  often  labeled  "cost-benefit" 
analyses,  and  vice-versa.  The  key  distinction  is  that  a 
benefit-cost  analysis  must  value  all  outcomes  in  eco- 
nomic (e.g.,  dollar)  terms,  including  lives  or  years  of 
life  and  morbidity,  whereas  a  cost-effectiveness  anal- 
ysis serves  to  place  priorities  on  alternative  expendi- 
tures without  requiring  that  the  dollar  value  of  life 
and  health  be  assessed. 

According  to  Klarman,  since  CEA  does  not  require 
valuing  life  and  health  in  monetary  terms,  the  dilem- 
mas of  valuation  that  arise  with  CBA  can  be  avoided 
by  substituting  CEA  (317).  When  we  substitute  CEA 
for  CBA,  though,  we  have  lost  the  tool  for  setting 
priorities  among  several  fields  of  activity. 

Richard  Layard  who  edited  a  major  volume  on 
CEA  and  CBA,  holds  that  CEA  is  useful  whenever 
CBA  becomes  impossible  (349).  The  reason  is  that 
even  if  the  planned  benefit  itself  cannot  be  valued 
sensibly,  it  is  still  useful  to  compare  the  costs  of  pro- 
viding the  same  benefit  in  different  ways.  Apart  from 
providing  a  value  for  the  benefits,  he  says,  the  pro- 
cedures involved  in  CEA  are  exactly  the  same  as 
those  for  CBA.  Self,  a  major  critic  of  the  cost-effec- 
tiveness technique,  says  that  cost-effectiveness  can 
easily  shade  into  CBA  because  goals  cannot  be  de- 
scribed with  precision  (556). 

In  general,  though,  CEA  aims  to  tell  policymakers 
the  least  expensive  way  of  achieving  an  already 
selected  goal,  whereas  CBA  seeks  to  help  them  select 
from  among  a  choice  of  goals.  Largely  because  it  is 
considerably  more  ambitious,  CBA  presents  more 
ethical  difficulties  than  CEA  (though  the  latter  is  cer- 
tainly not  free  of  them).  This  paper  concentrates 


largely  on  CBA,  but  much  of  what  is  said  is  equally 
applicable  to  CEA. 

Implications  of  the  Conflict  Between 
Equity  and  Efficiency 

Among  its  practitioners,  there  is  substantial  agree- 
ment that  efficiency  is  the  chief  value  underpinning 
economics.  Those  practitioners  also  freely  acknowl- 
edge that  since  this  value  is  its  central  one,  econom- 
ics— and  particularly  CBA — gives  short  shrift  to 
another  important  value:  equity.  The  conflict  be- 
tween equity  (often  expressed  by  the  terms  equality, 
justice,  fairness)  and  efficiency  is  an  enduring  one  in 
both  politics  and  in  economics.  This  conflict  is  the 
most  important  and  intractable  value  conflict  in  CBA 
and  is  also  its  most  important  ethical  issue.  How,  in 
pursuit  of  the  least  wasteful  way  of  spending  the  pub- 
lic dollar  for  a  given  commodity,  can  we  also  make  it 
available  to  everybody? 

Efficiency. — Economics  is  grounded  in  philoso- 
phy.3 Its  chief  underpinnings  came  out  of  the  philo- 
sophical school  known  as  utilitarianism.  Probably 
the  most  thoroughgoing  attempt  to  construct  a  moral 
theory  whose  chief  value  is  efficiency  was  made  in 
the  late  18th  century  by  Jeremy  Bentham  (49).  One  of 
the  earliest  utilitarians,  Bentham  intended  his  system 
to  apply  both  to  the  broad  social  arena  encompassing 
politics  and  legislation,  and  to  the  action  of  individ- 
uals. The  goal  of  Bentham's  utilitarianism  was  to 
maximize  pleasure  and  minimize  pain  for  all  who 
stood  to  be  affected  by  an  action. 

Bentham's  system  presupposes  an  ability  to  quan- 
tify pleasures  and  pains.  The  process  he  described  for 
judging  outcomes  required  a  precise  specification  of 
the  tendencies  of  actions  to  produce  pleasures  or 
pains.  All  such  tendencies  are  added  together,  and 
then  totaled  for  all  affected  individuals  until  an  op- 
timal "balance"  of  pleasures  over  pains  is  reached. 
Bentham's  scheme  is  sometimes  called  the  "hedonic 
calculus"  or  the  "felicific  calculus."  The  term  "calcu- 
lus" should  be  taken  quite  literally,  since  social  deci- 
sionmaking using  this  method  requires  straightfor- 
ward calculations — adding  and  subtracting  units  of 
pleasure  and  pain. 

Utilitarians,  after  Bentham,  subscribed  to  the  ba- 
sic idea  of  maximizing  utility  as  a  social  goal,  but' 
doubted  the  feasibility  of  constructing  cardinal  meas- 
ures of  utility.  They  sought  to  achieve  the  same  result 
by  employing  an  ordinal  scale.  Like  Bentham's,  their 
approach  still  allows  for  a  meaningful  notion  of  max- 
imizing individual  welfare  by  enabling  comparisons 


3  As  Kenneth  Boulding  has  pointed  out,  Adam  Smith  was  a  professor  of 
moral  philosophy  (67). 
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of  states  of  affairs  as  "better"  or  "worse,"  although  it 
is  not  committed  to  implausible  measurements  and 
calculations  of  pleasures  and  pains.  The  resort  to  or- 
dinal measurement  of  utility,  however,  founders  on 
the  nettlesome  problem  of  interpersonal  comparisons 
of  utility.  Until  we  work  out  a  common  measure  for 
one  person's  "better"  and  another's  "worse,"  the 
achievement  of  social  welfare  will  remain  an  elusive 
goal. 

Equity. — In  contrast  to  the  value  of  efficiency  that 
characterizes  utilitarian  theory,  the  central  value  in 
philosophical  theories  based  on  the  concept  of  justice 
or  fairness  is  equity.  John  Rawls's  theory  of  "justice 
as  fairness"  (494)  offers  a  compelling  alternative  to 
the  philosophical  foundations  of  CBA.  Rawls  criti- 
cizes utilitarianism's  willingness  to  allow  the  greater 
gains  of  some  to  offset  the  lesser  losses  of  others.  In 
theory,  all  versions  of  utilitarianism  could  justify  ac- 
tions by  a  majority  that  would  enhance  the  major- 
ity's position  at  the  expense  of  a  minority.  Even  if  it 
could  produce  a  total  social  welfare  based  on  ordinal 
measures  of  satisfaction,  however,  Rawls  would  still 
object  to  utilitarianism  on  the  ground  that  individual 
rights  and  liberties  are  liable  to  get  lost  in  the  aggre- 
gating shuffle.  Although  utilitarianism  presents  itself 
initially  as  an  individualistic  democracy  of  pleasures 
("everybody  to  count  for  one,  nobody  for  more  than 
one" — Bentham),  Rawls  observes,  aggregating  the 
desires  of  each  individual  into  a  total  system  of  desire 
actually  results  in  a  morally  defective  impersonality 
that  allows  some  people  to  be  used  merely  for  the 
benefit  of  others. 

Rawl's  rival  conception  of  equity  is  unique. 
Whereas  some  political  theorists  construe  equity  to 
mean  equality  (to  everyone  an  equal  share),  and 
others  treat  it  as  a  proportionality  of  some  sort  (to 
each  according  to  need,  merit,  contribution,  etc.), 
Rawls  begins  his  notion  of  distributive  justice  with  a 
preference  for  equality  that  is  immediately  hedged  by 
a  so-called  "difference  principle."  This  principle  per- 
mits significant  differences  in  wealth  and  social 
status,  but  only  those  differences  that  benefit  the 
least  advantaged  members  of  society.  (This  device 
makes  interpersonal  comparisons  of  utility  unneces- 
sary.) Thus,  while  it  it  permissible  for  a  society  to 
allow  certain  kinds  of  inequalities  that  raise  every- 
one's standard  of  living — in  fact,  Rawls  holds  that  it 
would  be  irrational  to  insist  on  equal  shares  of  fewer 
goods — the  difference  principle  rules  out  those  in- 
equalities that  improve  the  status  of  the  better-off  at 
the  expense  of  the  worse-off .  Once  a  society  adopted 
the  difference  principle,  individuals  would  be  entitled 
to  whatever  goods  or  status  they  could  obtain  or 
achieve  through  voluntary  transactions  on  the  free 
market,  gifts,  and  so  on. 


Two  examples  will  suffice  to  illustrate  the  sorts  of 
equity  problems  central  to  CBA:  1)  the  different 
ways  in  which  people's  lives  and  preferences  are  eval- 
uated, and  2)  the  way  in  which  the  future  is  evaluated 
compared  with  the  present. 

Valuing  Lives  and  Preferences 

The  problem  of  placing  different  values  on  differ- 
ent lives  is  keenly  felt  and  discussed  in  all  the 
theoretical  literature  on  CBA,  even  by  its  friendliest 
adherents.  This  problem  is  particularly  germane  to 
the  health  field  and  is  as  old  as  CBA  in  that  field.  The 
17th  century  estimates  of  the  value  of  life  by  Sir 
William  Petty  illustrate  a  problem  that  remains  un- 
solved (194).  To  strengthen  his  argument  about  what 
the  government  might  save  by  transporting  people 
out  of  the  city  and  saving  them  from  the  London 
plague,  Petty  valued  people  at  only  £20  each,  be- 
cause that  was  the  lowest  price  for  a  slave.  For  li is 
largely  illustrative  and  rhetorical  purposes,  setting 
the  lowest  possible  valuation  on  labor  effectively 
made  the  point.  Petty's  calculations  of  what  the  gov- 
ernment might  have  saved  was  an  underestimate;  the 
actual  amount  would,  of  course,  be  much  more, 
since  not  only  "slaves"  would  be  transported  out  of 
the  city  and  saved  from  the  plague,  but  "more  valu- 
able" persons  as  well. 

In  a  19th  century  sanitation  study  in  Massachu- 
setts, Lemuel  Shattuck  valued  adult  men  differently 
from  women  and  children  (559)— a  practice  often  ad- 
hered to  today.  Shattuck's  estimates  of  the  benefits  of 
better  sanitation  in  Massachusetts  involved  labor 
costs  (calculated  at  an  average  of  $50  a  year)  totaling 
almost  $5.5  million,  to  which  he  added  the  cost  of 
public  support  of  dead  laborers'  widows  and  chil- 
dren. But  the  latter  costs  came  to  only  an  additional 
estimated  $2  million. 

The  Issue  and  the  Methods 

Fein  puts  the  whole  issue  nicely  (194): 

Does  the  measurement  of  a  person's  worth  in  terms 
of  his  productive  contribution  really  represent  our  so- 
cial values?  I  believe  that  it  does  not  do  so.  In  particu- 
lar, it  fails  adequately  to  take  account  of  equity  and 
distributional  considerations  (which  many  believe  to 
be  one  of  the  major  functions  of  government). 
Fein  points  out  that  taking  account  of  individual 
characteristics  such  as  potential  earnings  could  lead 
society  to  provide  health  care  services  to  those  with 
the  highest  potential  income  and  to  direct  them  away 
from  those  with  low  earning  capacity  and  less  educa- 
tion. Females  in  such  a  scheme  are  at  a  disadvantage 
compared  to  males,  as  are  the  old  compared  to  the 
young,  and  blacks  compared  to  whites.  In  each  case, 
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the  disadvantaged  individuals  are  the  ones  with  the 
lower  incomes. 

Klarman  argues  that  some  elements  of  the  contro- 
versy over  valuing  lives  have  been  settled  by  recent 
technical  developments  (317).  Whereas  15  years  ago 
a  wife  not  employed  outside  the  home  did  not  count 
for  very  much  in  a  CBA,  he  notes,  recently  house- 
wives' services  have  been  given  a  monetary  value. 
The  lives  of  housewives  are  now  usually  figured  on 
that  basis,  though  still,  of  course,  at  a  lower  rate  than 
the  lives  of  their  husbands  employed  outside  the 
home. 

When  cost-benefit  calculations  were  begun  at 
HEW  in  the  middle  1960's,  Agency  officials  realized 
that  the  benefits  calculation  discriminated  against 
women  and  the  elderly  (265).  According  to  Grosse, 
there  were  fears  among  some  that  too  vigorous  pur- 
suit of  the  underlying  logic  would  penalize  not  only 
health  programs  for  the  aged  (medicare  was  just  get- 
ting started),  but  programs  for  the  poor,  as  well.  He 
notes,  however,  that  (265): 

In  actual  practice,  for  the  particular  programs  stud- 
ied, these  concerns  were  not  critical.  The  programs  for 
cervical  and  breast  cancer  looked  to  be  good  despite 
their  being  for  women.  As  for  the  poor,  most  of  the 
programs  considered,  especially  cervical  cancer,  syph- 
ilis and  tuberculosis,  were  aimed  primarily  at  them, 
and  projects  were  usually  located  to  serve  low-income 
residents. 

By  implication  at  least,  this  is  a  way  to  get  around 
some  of  the  distributional  problems  inherent  in  CBA: 
If  decisionmakers  are  committed  to  just  distribution 
of  resources  and  are  interested  in  choosing  the  best 
programs  aimed  chiefly  at  the  poor,  i.e.,  if  equity 
considerations  are  built  in  from  the  start,  then  the 
problem  of  equity  can  be  dealt  with.  This  is,  of 
course,  a  big  if. 

Mishan  believes  that  although  distributional  and 
equity  considerations  do  offer  grounds  for  objection 
to  CBA,  the  objection  is  not  a  strong  one  (410).  Even 
if  a  project  is  cost-beneficial,  Mishan  notes,  it  can 
always  be  rejected  on  grounds  of  inadequate  atten- 
tion to  equity.  Writing  on  the  relationship  of  Chris- 
tian ethics  to  CBA,  Denys  Munby  agrees,  arguing 
that  the  use  of  differential  income  data  usually  makes 
social  decisions  easier  (416).  If  one  found  out  that 
only  millionaires  would  be  affected  by  the  destruc- 
tion of  a  particular  beauty  spot,  one  might  not  worry 
very  much  about  its  destruction.  "What  is  quite  clear 
is  that  we  can  make  our  analysis  as  egalitarian  as  we 
want,"  he  says.  What  these  views  require,  however, 
is  that  policymakers  be  committed  to  equity — and 
that  they  act  on  the  basis  of  that  commitment. 

Another  major  objection  to  the  usual  way  people 
are  valued  for  purposes  of  CBA  is  that  focusing  on 
income  ignores  other  indices.  Mishan  acknowledges 


that  CBA  tends  to  ignore  indicators  of  social  merit, 
but  argues  that  we  have  no  good  way  now  of  judging 
their  worth  (410). 

CEA  attempts  to  avoid  the  seemingly  insoluble 
valuation  problem  of  CBA  by  simply  jettisoning 
much  of  it.  Instead  of  comparing  costs  in  dollars  to 
benefits  in  dollars,  CEA  compares  costs  in  dollars  to 
benefits  expressed  in  nonmonetary  terms  (e.g.,  num- 
bers of  deaths  averted).  As  Weinstein  and  Stason 
note,  however,  a  limitation  of  both  CEA  and  CBA  is 
(627): 

.  .  .  that  the  benefits  and  costs  to  individual  mem- 
bers of  society  need  to  be  aggregated.  If  the  inequita- 
ble distribution  of  benefits  and  costs  across  individu- 
als or  groups  are  of  concern,  a  single  cost-effectiveness 
measure  will  not  do.  However,  as  economists  are 
wont  to  argue,  over  large  number  of  programs  and 
practices  the  inequities  are  likely  to  even  themselves 
out  and,  with  some  exceptions,  may  reasonably  be  ig- 
nored. 

Even  assuming  that  some  of  the  aforementioned 
problems  could  be  solved,  other  problems  in  valua- 
tion would  remain.  In  comparing  the  loss  of  a  man's 
life  with  that  of  a  woman,  one  is  at  least  comparing 
two  identical  types  of  outcome.  As  Fein  points  out, 
however,  problems  arise  in  comparing  apples-and- 
oranges  types  of  outcomes,  for  example,  lives  saved 
in  comparison  with  blindness  prevented  (194). 
Grosse  agrees,  noting  that  when  dealing  only  with 
the  cost  per  death  averted,  there  is  no  way  to  com- 
pare or  rank  diseases  that  do  not  usually  kill  (265). 

CBA  has  not  yet  found  a  common  way  of  measur- 
ing different  outputs  from  different  programs  so  that 
they  can  be  sensibly  compared.  Finding  a  common 
unit  of  measurement  is  very  difficult.  We  are  unable 
to  measure  units  of  satisfaction  or  of  happiness  gen- 
erated by  various  government  activities,  nor  are  we 
able  to  compare  one  person's  satisfactions  with 
another's. 

One  way  of  dealing  with  this  problem  has  been  by 
using  the  concept  of  quality-adjusted  life  years 
(QALYs).  Even  the  concept's  chief  proponents,  how- 
ever, concede  that  the  weights  it  gives  to  varying 
states  of  health  are  subjective,  so  much  so  that  (627): 
In  a  cost-effectiveness  analysis  at  the  societal  level, 
it  is  therefore  essential  that  a  range  of  possible  weights 
be  used  to  reflect  the  spectrum  of  individual  val- 
ues .  .  .  Most  analyses  avoid  quality-of-life  considera- 
tions entirely,  quantifying  only  the  change  in  life  ex- 
pectancy. Where  the  quality-of-life  effects  are  believed 
to  be  important,  however,  the  credibility  of  an  other- 
wise effective  analysis  may  be  jeopardized.  Tradeoffs 
involving  quality-of-life  considerations  are  made  im- 
plicitly by  health-care  decisionmakers  all  the  time;  the 
role  of  the  analysis  is  to  make  them  explicit. 
The  difficulty  here,  as  the  authors  implicitly  ac- 
knowledge when  they  draw  attention  to  the  subjec- 
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tive  nature  of  the  QALY  concept,  is  that  such  assess- 
ment reintroduces  the  problem  of  determining  values 
that  the  use  of  CEA  is  supposed  to  avoid. 

Mishan's  text  contains  extensive  material  on  the 
valuation  of  life  and  limb  (410).  Though  philoso- 
phers could  legitimately  disagree  with  his  view, 
Mishan  regards  saving  life  as  symmetrical  with  losing 
it,  so  he  bases  his  techniques  almost  exclusively  on 
established  legal  methods  of  compensating  victims. 
Self  says  this  is  actually  an  ethical  rather  than  a  legal 
position,  a  position  that  specifies  that  involuntary 
losses  should  be  compensated  (556). 

Despite  repeated  expressions  of  dissatisfaction 
with  it,  the  most  common  way  of  calculating  the 
worth  of  a  person's  life  is  by  discounting  to  the  pres- 
ent that  person's  expected  future  earnings.  Mishan 
discusses  alternatives  to  this  conventional  way  of 
valuing  loss  of  life,  one  being  to  perform  auxiliary 
calculations  that  take  account  of  a  victim's  suffering 
or  family's  bereavement  (410).  He  appears  to  favor 
evaluating  the  saving  of  life  by  referring  to  what  each 
person  would  be  willing  to  pay  or  to  receive  for  an 
estimated  change  in  the  risk  to  his  or  her  life.  These 
calculations  could  change  over  time.  Mishan  argues 
that  one  might  ultimately  want  to  forgo  bereavement 
calculations  altogether  because  of  the  gradual  loosen- 
ing of  family  ties  and  the  decline  of  emotional  inter- 
dependence. In  a  wholly  impersonal  society,  any 
member  of  the  community  is  equally  replaceable — in 
such  a  society  there  would  be  no  suggestion  of  com- 
pensating for  bereavement. 

Weighting  Schemes 

One  attempt  to  improve  equity  considerations  has 
been  to  use  utility  rather  than  dollars,  and  to  assume 
that  income  has  diminishing  marginal  utility.  This 
means  each  person's  income  is  valued  equally  up  to  a 
fixed  amount;  income  over  that  amount  is  given  less 
importance  in  the  calculations.  One  obvious  difficul- 
ty with  such  weighting  attempts  is,  as  Mishan  (410) 
and  others  (194,556,349)  point  out,  to  secure  a  con- 
sensus on  the  weighting.  And,  of  course,  a  weighting 
scheme  could  still  end  up  making  the  rich  richer  and 
the  poor  poorer. 

Klarman  argues  that  weighting,  though  not  wrong, 
is  certainly  judgmental  (317).  To  make  his  point, 
Klarman  cites  as  an  example  the  weighting  of  earned 
income  more  heavily  than  the  same  amount  of  mon- 
ey received  from  public  assistance.  Such  weighting  is 
based  on  the  assumption  that  money  earned  from 
work  is  somehow  morally  better  than  money  re- 
ceived as  public  largesse.  It  may  be  legitimate  to  use 
such  weighting  in  CBA,  but  as  Klarman  states  (317): 
What  must  be  recognized  is  that  weights  are  judg- 
mental, are  likely  to  be  arbitrary  (at  least  initially), 


should  be  derived  in  the  public  arena,  and,  above  all, 
must  be  clearly  stated. 

Layard  favors  CBA  in  which  valuation  is  arrived 
at  by  asking  people  what  they  would  pay  to  acquire 
the  benefits  or  avoid  the  costs  (349).  Of  course,  one 
doesn't  usually  ask  people  such  questions  directly, 
although  some  economists  advocate  a  questionnaire 
approach.  Far  more  often,  values  are  put  on  such 
programs  indirectly  by  inferring  what  people  value 
from  their  behavior,  often  from  what  they  buy. 

Munby  approves  attempts  at  valuation  based  on 
people's  purchasing  behavior  (416): 

In  principle,  the  whole  analysis  is  entirely  demo- 
cratic; it  starts  and  ends  with  what  people  actually 
want,  as  shown  by  their  actual  choices. 
On  the  other  hand,  though,  as  Self  argues,  willing- 
ness to  pay  is  in  some  sense  related  to  ability  to  pay, 
so  analyses  in  which  this  is  used  for  the  purpose  of 
valuation  are  still  biased  toward  the  better-off  (556). 

The  Veatch  Critque 

Robert  Veatch  asserts  that  valuing  lives  presup- 
poses that  we  have  decided  the  question  of  value  to 
whom  (603).  In  some  theories  of  valuing  lives,  the 
relevant  consideration  is  assumed  to  be  value  to 
society  or  aggregate  value.  In  others,  it  is  value  to  the 
individual  (e.g.,  self-rating  weighting  systems),  to 
some  specific  social  group,  or  to  the  person  who  pays 
the  bills  (who  might  also  be  the  U.S.  taxpayer). 

Veatch  also  points  to  a  problem  with  the  willing- 
ness-to-pay approach  for  weighting  (603): 

At  least  up  to  a  certain  point,  the  older  one  be- 
comes, the  more  valuable  life  becomes  (because,  sub- 
ject to  willingness  to  borrow,  one  is  generally  willing 
to  pay  more  to  avoid  death  risks  since  one  has  a 
higher  income  level).  On  the  other  hand,  in  human 
capital  terms,  the  older  one  becomes,  the  less  valuable 
life  is,  to  the  point  that  at  retirement  (at  least  for  males 
who  tend  not  to  be  involved  in  non-monetized  domes- 
tic labor),  life  is  "worthless".  .  .  Any  formula  based 
on  personal  estimates  of  how  much  one  would  pay  to 
avoid  a  certain  risk  of  death  accepts  the  status  quo  dis- 
tribution of  incomes  in  our  society.  Accepting  it  may 
be  efficient — it  may  be  utility  maximizing — yet  I  am 
still  convinced  it  is  not  just  and  it  is  not  right. 
Veatch  argues  that  the  Kantian  maxim  that  the  in- 
dividual is  to  be  treated  as  an  end,  and  never  only  as 
a  means,  prohibits  policies  in  which  one  individual  or 
group  benefits  at  the  expense  of  another,  even  if  the 
net  benefits  far  outweigh  the  harms.  Many  econo- 
mists have  dealt  with  this  and  similar  arguments  by  a 
pragmatic  counterargument — that  we  are  in  fact 
making  precisely  such  invidious  comparisons  with 
current  policies  arrived  at  on  intuitional  or  other 
"nonscientific"  grounds.  By  this  argument,  an  impor- 
tant advantage  of  valuing  lives  for  purposes  of  CBA 
is  that  it  brings  us  face-to-face  with  the  fact  that,  even 
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in  a  supposedly  egalitarian  society  like  the  United 
States,  we  value  different  people's  lives  differently. 
As  Veatch  says,  however  (603): 

If  it  is  conceptually  unsound  to  equate  the  real  value 
of  a  life  with  the  value  of  the  labor  that  life  will  pro- 
duce, it  is  also  unsound  to  conclude  that  there  is  any 
necessary  relationship  between  what  I  would  pay  for  a 
program  to  add  years  of  life,  and  what  society  ought 
to  pay  for  that  program. 
Willingness-to-pay   approaches  favor   those  with 
greater  ability  to  pay.  Even  willingness-to-pay  meas- 
ures that  concentrate  on  percentage  of  income  rather 
than  absolute  dollars  really  do  not  solve  the  problem, 
because  selection  of  an  acceptable  percentage  of  in- 
come also  has  a  very  strong  relationship  to  income 
level. 

Finally,  willingness-to-pay  measures  fail  to  tell  us 
anything  about  people's  attitudes  toward  programs 
in  which  they  have  no  particular  stake,  like  prenatal 
diagnosis  for  women  who  have  no  plans  to  bear  chil- 
dren and  are  not  close  to  anybody  who  has  such 
plans.  Sheer  self-interest  dictates  that  one  should  be 
willing  to  pay  for  programs  in  which  one  is  or  may  be 
personally  involved.  As  Veatch  notes  (603): 

One  might  have  a  self  interest  in  avoiding  the  wit- 
nessing of  suffering  around  him,  but  this  does  not 
seem  to  be  the  real  moral  basis  of  a  commitment  to 
help  programs  designed  to  affect  a  condition  one  is 
virtually  certain  not  to  have. 
Veatch  here  ignores  the  argument  (based  on  CBA!) 
that  such  programs  will  save  taxpayers  money  (e.g., 
by  reducing  the  cost  of  institutions  for  the  retarded). 

Veatch  identifies  four  different  assumptions  that 
can  be  used  as  the  basis  for  valuing  lives:  1)  all  lives 
are  of  equal  value,  2)  the  prime  of  life  is  most  worth- 
while (because  it  is  most  productive),  3)  youth  is 
most  worthwhile  (because  young  people  have  more 
life  ahead  of  them),  4)  old  age  is  most  worthwhile 
(because  older  people  are  wiser?).  Still  Veatch  says 
(603): 

...  all  have  an  equal  claim  to  the  health  care 
needed  to  provide  a  level  of  health  equal,  insofar  as 
possible,  with  other  people's  health  ...  it  will  neces- 
sarily have  to  give  way  to  other  claims  upon  occasion, 
but  it  should  be  the  decisive  starting  point  for  evaluat- 
ing alternative  public  policy. 
Thus,  social  usefulness,  willingness-to-pay,  future 
productivity,  and  other  such  measures  are  and  ought 
to  be  irrelevant. 

CBA  is  all  right  for  some  things,  Veatch  says,  like 
telling  us  that  some  diseases  kill  people  in  the  prime 
of  life.  Whenever  policy  choices  conflict  with  indi- 
vidual rights  and  justice,  however,  Veatch  wishes  to 
promote  justice  "even  if  that  means  lower  aggregate 
indicators  of  utility"  (603).  Younger  people  should 
receive  a  higher  priority  than  others,  because  "an 


essential  part  of  the  egalitarian  principle  of  justice  is 
that  there  should  be,  insofar  as  possible,  an  equal  op- 
portunity to  live  to  the  same  age  as  others."  There 
should  also  be  equality  of  opportunity  to  live  life 
without  suffering,  Veatch  believes.  But  here  the 
policy  objective  should  not  be  to  achieve  a  maximum 
reduction  in  suffering  (because  that  is  an  aggregate 
measure),  but  rather  to  reduce  those  conditions  that 
produce  the  greatest  suffering  in  the  individuals  who 
have  them.  Veatch  believes  that  sometimes  the  needs 
of  the  poor  may  have  to  be  placed  ahead  of  those 
who  may  die  relatively  early.  He  also  favors  ex- 
cluding or  giving  low  priority  to  deaths  for  which  the 
victim  is  personally  responsible  (e.g.,  death  from 
lung  cancer  caused  by  cigarette  smoking).  According 
to  Veatch,  involuntary  conditions  should  have 
higher  priority  than  voluntary  ones. 

Discounting 

Another  important  distributional  problem  that  re- 
quires at  least  a  brief  discussion  is  selection  of  the  dis- 
count rate  and  the  related  problem  of  comparing  the 
future  with  the  present,  given  inflation  and  the  fact 
that,  in  general,  people  prefer  benefits  now  to  later. 
Self  traces  the  problem,  in  part,  to  the  Pareto  princi- 
ple itself,  which  he  believes  is  "strongly  biased 
towards  the  status  quo"  (556). 

Layard  (349)  points  out,  and  other  analysts  (317, 
194)  agree,  that  the  question  of  the  welfare  of  future 
generations  is  simply  ignored  in  most  CBAs.  Layard 
believes  most  economists  would  argue  that  projects 
should  be  judged  exclusively  in  terms  of  their  effects 
on  the  welfare  of  those  now  living.  He  notes  (349): 
If  one  takes  the  alternative  view  that  cost-benefit 
analysis  aims  to  throw  light  on  what  is  right,  it  is  diffi- 
cult to  think  of  any  ethical  justification  for  ignoring 
future  generations.  A  practical  argument  is  sometimes 
put  forward  for  ignoring  them — that  we  cannot  know 
their  preferences.  However,  there  are  many  items  (like 
life)  where  we  do  not  know  how  they  are  valued  by 
present  generations,  and  many  (like  bread)  that  we 
can  be  fairly  sure  what  future  generations  will  feel. 
The  discount  rate,  which  takes  into  account  what  a 
dollar  invested  today  would  earn  in  interest  if  it  were 
not  being  spent  on  health  care,  tends  to  devalue  the 
future.  MacRae  says  economists'  evaluations  of  the 
future  may  be  wrong  (394).  Rawls  has  even  suggested 
a  zero  discount  rate  to  promote  justice  among  gener- 
ations (494).  In  his  discussion  of  the  discount  rate, 
Mishan  describes  the  political  constraints  on  deci- 
sions about  which  discount  rates  to  use  (410).  Klar- 
man  notes  that  Federal  agencies  have  traditionally 
used  a  wide  range  of  discount  rates,  usually  without 
giving  any  kind  of  justification  (317).  The  conse- 
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quences  of  choosing  a  high  or  a  low  rate,  he  says,  are 
clear:  Low  rates  favor  projects  or  programs  whose 
benefits  accrue  in  the  distant  future.  When  the  proj- 
ect is  short-lived,  the  choice  of  a  discount  rate  is  mi- 
nor. Nevertheless,  Klarman  says,  there  may  even  be 
some  merit  in  employing  a  single  number  for  all  pub- 
lic human  investment  projects. 


Conclusions 

The  distributional  issues  discussed  above  present 
both  philosophical  and  practical  problems  for  econo- 
mists. According  to  Layard,  most  economists  would 
argue  that  income  should  be  redistributed  by  direct 
cash  transfer  rather  than  by  juggling  choices  of  public 
projects  (349).  But  Layard  believes  that  if  we  are 
reasonably  certain  that  the  cash  will  not  be  so  distrib- 
uted, then  the  poor  person's  dollar  ought  to  be  val- 
ued more  highly  than  the  rich  person's. 

As  he  sees  it,  the  problem  is  less  one  of  equity  than 
it  is  of  knowing  which  constraints  should  be  taken  as 
given  and  figured  into  the  analysis.  If  a  Government 
agency  knows  for  certain  that  cash  will  not  be  redis- 
tributed, even  if  it  should  be,  then  the  agency  should 
allow  for  distributional  factors  when  it  evaluates  a 
project.  It  should  not  allow  for  those  factors  if  it  can 
ensure  that  redistribution  will  be  achieved  by  some 
more  appropriate  method.  In  practice,  however,  a 
Government  agency  cannot  know  this.  Until  this  is 
settled,  it  may  be  impossible  to  rationally  appraise  a 
project.  Privately  employed  economists,  too,  are  free 
to  incorporate  distributional  measures  (e.g.,  no-cost 
transportation  to  neighborhood  health  centers  for  the 
elderly  )  into  their  equations. 
According  to  Layard  (349): 

A  decision-maker  should  choose  from  his  available 
set  of  actions  the  ones  which  maximize  social  welfare, 
subject  to  all  the  constraints  over  which  he  has  no  con- 
trol. 

It  should  be  noted,  however,  that  this  approach 
allows  economists  enormous  latitude  without  requir- 
ing that  they  incorporate  distributional  considera- 
tions. It  can  also  be  argued  that  such  an  approach 
completely  subverts  the  purpose  of  CBA,  which  at- 
tempts to  free  the  policy  process  from  the  individual 
beliefs  and  biases  of  the  policymaker. 

To  summarize,  the  traditional  approach  to  CBA 
excludes  formal  consideration  of  distributional  ef- 
fects such  as  equity  and  fairness.  Since  economists 
disagree  about  how  to  solve  this  problem,  equity 
considerations  are  likely  to  continue  to  be  underem- 
phasized  in  practice.  Mishan,  for  one,  believes  that 
the  situation  is  unlikely  to  change  in  the  foreseeable 
future (410). 


Implications  of  Data  Problems 

After  distributional/equity  issues,  the  criticism 
most  often  made  of  CBA  is  that  the  data  employed 
are  simply  not  reliable.  This  criticism  appears  to  be 
legitimate  to  an  undetermined,  possibly  large,  extent. 
It  is  a  very  serious  criticism,  chiefly  because  it  con- 
flicts with  the  quantitative  and  scientific  trappings  of 
CBA. 

The  general  problem  here,  as  Parish  points  out,  is 
that  CBA  is  a  form  of  modeling,  and,  like  all  other  at- 
tempts at  modeling,  expresses  only  certain  aspects 
(and  those  perhaps  not  wholly  accurately)  of  a  very 
complex  world  (475).  CBA  is  "irretrievably"  second- 
best;  that  is,  CBA  seeks  to  explore  the  consequences 
of  particular  actions  in  the  imperfect  economy  as  it 
exists  and  in  which  the  insights  of  theories  applicable 
to  a  first-best  world  may  be  misleading.  According  to 
Parish,  "Its  practitioners  are  frequently  tempted  to 
simplify  their  problem  by  making  first-best  assump- 
tions .  .  ."  (475). 

The  Problem  of  Defining  Outcomes  and  Goals 

Criticisms  of  data  are  made  on  a  number  of  dif- 
ferent grounds.  Most  obvious  are  the  simple  inade- 
quacies of  our  current  reporting  and  data  systems.  In 
a  very  real  sense,  of  course,  such  inadequacies  are 
technical  problems  that  can  be  improved  once  they 
are  identified  and  someone  in  a  position  of  power 
decides  to  deal  with  them  and  improve  the  system. 

Fein,  however,  has  identified  a  number  of  much 
more  difficult  data  problems  (194).  One  concerns  the 
measurement  of  outcomes  which  are  conceptually 
amorphous  (e.g.,  higher  levels  of  health)  and  to 
which  many  factors  contribute  (e.g.,  housing,  in- 
come, nutrition,  environment,  and  medical  care  of 
all  kinds).  In  addition,  there  are  many  factors  whose 
relative  contribution  may  differ  for  different  persons, 
and  whose  proportional  importance  is  largely  un- 
known. Another  problem  is  how  to  measure  the 
goals  of  a  health  program  that  exist  on  a  continuum. 
It  is  more  difficult  to  measure  continuous  states  with 
a  wide  range  of  effect,  like  pain  or  impairment  of 
functional  ability,  than  to  measure  discontinuous 
states  like  life  and  death.  Furthermore,  difficulties  in 
measurement  create  a  bias  in  favor  of  programs  that 
have  easily  measurable  goals. 

Most  discussions  of  CBA  mention  the  selection  of 
goals  and  objectives  as  a  problem.  Many  critics  and 
even  some  proponents  of  CBA  feel  that  the  goals  are 
often  vague  and  nonspecific  (e.g.,  "health").  Kenneth 
Boulding  points  out  that  we  often  agree  on  major 
goals  (67).  We  disagree  on  how  to  reach  the  goals, 
and  in  some  cases,  on  what  the  alternative  ap- 
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proaches  might  be.  Mishan,  for  example,  criticizes 
Klarman's  study  of  syphilis  by  pointing  out  that  the 
calculations  were  based  on  the  goal  of  eradicating  the 
disease  (410).  There  was  no  comparison  with  a  defin- 
ed control  program  to  reduce  the  disease  to  some  pre- 
specified  level. 

Ironically,  Klarman  himself  is  quite  aware  of  the 
difficulty  (317).  He  believes  the  costs  of  a  particular 
health  program  can  be  estimated,  but  that  the  field's 
chief  difficulty  arises  in  formulating  the  contents  and 
expected  outcomes  of  programs  before  estimating 
their  cost.  In  a  study  of  the  end-stage  renal  disease 
program,  for  example,  CEA  pointed  to  the  superiori- 
ty of  kidney  transplantation,  with  dialysis  only  for 
initial  and  backup  support.  Klarman  believes  that  if  a 
CBA  had  been  performed,  the  shortage  of  suitable 
kidneys  and  the  relative  ease  with  which  dialysis  fa- 
cilities could  be  expanded  might  have  yielded  a 
higher  net  benefit  value  for  dialysis. 

Economists  may  also  make  naive  assumptions 
about  the  practicality  of  stated  goals  and  outcomes, 
Klarman  says  (317).  At  least  in  the  early  days  of 
CBA,  he  notes,  there  was  a  tendency  by  economists 
to  attribute  greater  efficacy  to  medical  care  than  we 
now  believe  is  warranted. 

The  Problem  of  Uncertainty  About  the  Future 

The  whole  area  of  uncertainty  about  the  future 
— guesswork  about  costs  and  benefits  of  technologi- 
cal innovations  and  unanticipated  shifts  in  demand — 
continues  to  plague  CBA.  According  to  Mishan 
(410): 

The  problem  of  how  to  make  decisions  in  any  situa- 
tion where  the  past  affords  little  or  any  guidance  is  not 
one  that  can  be  satisfactorily  resolved  either  by  logic 
or  empiricism,  and  what  moves  have  been  formulated 
are  either  of  limited  application  or  of  no  practical 
value. 

To  cope  with  uncertainty,  one  can  figure  an  arbi- 
trary cutoff  period,  adjust  the  discount  rates,  or  sim- 
ply pick  a  subjective  probability.  A  number  of  recent 
articles  (627,564)  recommend  using  sensitivity  anal- 
ysis (627): 

...  In  this  method,  the  most  uncertain  features  and 
assumptions  in  the  cost-effectiveness  calculation  are 
varied  one  at  a  time  over  the  range  of  possible  values. 
If  the  basic  conclusions  do  not  change  when  a  particu- 
lar feature  or  assumption  is  varied,  confidence  in  the 
conclusion  is  increased.  If,  instead,  the  basic  conclu- 
sions are  sensitive  to  variations  in  a  particular  feature 
or  assumption,  further  research  to  learn  more  about 
that  feature  may  be  especially  valuable  .  .  . 

.  .  .  Examples  of  sensitivity  analyses  that  are  often 
useful  are  to  vary  the  estimates  of  the  degree  of  clini- 
cal efficacy  of  the  procedure  in  question,  to  vary  the 
weights  assigned  to  various  quality  levels  in  comput- 


ing quality-adjusted  life  expectancy,  and  to  test  a 
range  of  discount  rates,  say,  from  0  to  10  percent  per 
year. 

The  Problem  of  Using  Proxy  Goals 
and  Measures 

Another  problem  comes  from  the  use  of  proxy 
goals  and  measures  that  may  bias  the  number  and  va- 
riety of  options  for  action.  Fein's  cites  as  an  example 
measuring  the  health  of  children  by  counting  school 
days  missed  (194).  A  program  designed  to  improve 
children's  health  so  that  they  miss  fewer  days,  he 
argues,  is  different  from  one  that  focuses  so  heavily 
on  reducing  days  of  absence  that  its  success  results  in 
sick  children's  being  sent  to  school. 

In  fact,  some  argue  that  one  of  the  deficiencies  of 
CBA  is  that  it  is  a  complete  exercise  in  proxy  meas- 
ures, since,  as  Fein  himself  points  out,  monetary 
benefits  are  only  a  part  (and  in  some  cases  a  small 
part)  of  all  benefits,  and  they  do  not  represent  a 
stable  or  constant  fraction  of  all  benefits.  This  prob- 
lem has  been  obvious  at  least  since  Lemuel  Shattuck's 
1850  sanitation  study  in  Massachusetts  (559).  By 
spending  $3,000  on  sanitation  planning  and  technical 
assistance,  he  argued,  the  State  would  ultimately 
gain  at  least  $7.5  million  in  productivity  for  the  6,000 
unnecessary  deaths  averted  by  efficient  sanitation. 
But  Shattuck  also  understood,  and  said,  that  eco- 
nomic benefits  were  not  the  only  ones  that  sanitation 
measures  would  yield.  In  addition,  such  measures 
would  increase  public  happiness  (194).  CEA  is  an 
attempt  to  sidestep  such  problems,  particularly 
through  the  concept  of  QALYs  (627,563). 

For  Fein,  the  choice  of  data  is  very  important  be- 
cause budget  officials  will  tend  to  focus  on  those  out- 
comes that  have  numbers,  or  more  specifically,  eco- 
nomic values,  attached  to  them  (194).  Programs  that 
affect  future  productivity  thus  come  to  be  overvalued 
because  they  can  be  quantified,  and  programs  that 
relieve  pain  and  suffering  but  do  not  affect  produc- 
tivity come  to  be  undervalued.  According  to  Fein 
(194): 

It  may,  indeed,  be  that  programmes  addressed  to 
disabling  conditions  and  to  disease  involving  mortali- 
ty, rather  than  to  conditions  that  do  not  remove  the 
person  from  economic  activity,  should  be  favored. 
That  conclusion,  however,  should  not  be  reached  pri- 
marily because  some  things  can  be  measured  while 
others  cannot.  The  analyst  may  discount  the  nature  of 
the  difficulty  and  the  likelihood  that  this  might  occur, 
believing  that  his  description  of  the  items  (particular- 
ly, benefits)  that  cannot  be  measured  will  suffice  to 
alert  the  decision-maker  to  the  inadequacy  of  the 
numbers.  I  suggest,  however,  that  the  analyst  may 
underestimate  the  problem.  He  would  do  well  to  con- 
sider how  compelling  numbers  are  to  finance  officials 
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and  how  high  a  rate  of  discount  is  applied  to  words, 
however  well-turned  the  phrases  may  be. 
In  an  article  on  the  Christian  ethics  of  CBA,  Denys 
Munby  argues  that  the  technique  attempts  (416): 
.  .  .  precisely  to  take  account  of  all  the  "social 
costs"  that  social  reformers  have  (rightly)  accused  the 
pure  market  economies  of  ignoring  ....  Money  is 
merely  an  abstract  measuring  rod;  there  is  no  other  so 
comprehensive.  It  is  not  the  use  of  money  symbols 
that  can  be  criticized,  but  the  actual  valuations  have 
to  be  examined  to  see  whether  they  correspond  to  the 
way  we  think  things  should  be  valued. 

The  Problem  of  Deciding  on  Inclusions  and 
Exclusions  in  an  Analysis 

Klarman  points  out  the  great  difficulty  in  deciding 
which  factors  to  include  and  which  to  exclude  in  an 
analysis,  and  suggests  that  this  problem  is  particular- 
ly acute  in  the  health  field  (317).  Other  analysts 
agree.  Grosse  notes  that  the  HEW  study  of  the  costs 
of  automobile  accidents  and  their  prevention  did  not 
calculate  the  cost  of  seat  belts  (265),  but  he  does  not 
specify  why.  That  particular  omission  seems  very 
puzzling.  Similarly  not  figured  into  the  analysis  was 
the  possibility  that  lower  auto  insurance  rates  for 
everybody  could  result  from  a  successful  injury 
prevention  program. 

In  genetic  screening,  where  many  CBAs  have  been 
performed,  similar  puzzlements  crop  up.  A  number 
of  cost-benefit  studies  have  been  conducted  on  new- 
born screening  for  phenylketonuria  (PKU),  a  very 
rare  metabolic  disorder  leading  to  severe  mental  re- 
tardation (79).  Mental  retardation  can  be  partially 
prevented  by  putting  babies  with  PKU  on  a  special 
diet  shortly  after  birth.  On  paper,  PKU-screening 
programs  for  newborns  have  been  demonstrated 
over  and  over  again  to  save  society  money  by  reduc- 
ing the  number  of  retarded  people  who  need  to  be  in- 
stitutionalized. 

But  not  one  of  the  analyses  of  PKU-screening  pro- 
grams has  included  in  its  calculations  a  major  long- 
term  cost — that  of  finding  and  putting  back  on  the 
special  diet  during  pregnancy  young  women  with 
PKU  whose  children  will  otherwise  be  severely  re- 
tarded. Formerly,  women  with  PKU  lived  in  institu- 
tions and  did  not  reproduce.  As  a  result  of  the  special 
diet  initiated  after  birth,  today  many  women  who 
were  born  with  the  disorder  lead  normal  lives  and 
have  children. 

The  cost  of  a  20-year  or  longer  followup  for  these 
women  is  enormous,  and  yet  it  is  justified  on  both 
practical  and  ethical  grounds.  Having  saved  young 
women  born  with  PKU  from  severe  retardation,  the 
state  wants  to  prevent  retardation  in  their  offspring 
for  which  it  might  otherwise  have  to  bear  financial 


responsibility.  In  addition,  it  can  certainly  be  argued 
that  Government  also  has  a  moral  obligation,  once  a 
public  policy  has  been  instituted,  to  help  those  people 
who  are  affected  deal  with  new  problems  that  arise  in 
their  lives  as  a  result  of  State  intervention. 


The  Problem  of  Infinite  Externalities 

Boulding  points  out  that  almost  everything  we  do 
turns  out  differently  from  what  we  expect;  both  the 
bad  and  good  are  often  unintentional  (67).  So,  it  is 
not  surprising  that  a  major  problem  with  assessing 
cost  and  benefits  are  so-called  externalities,  otherwise 
known  as  external  effects,  neighborhood  effects,  side 
effects,  spillover  effects,  or  spillovers.  Externalities 
include  such  things  as  the  effect  of  building  a  road  on 
esthetic  sensibilities,  on  noise  and  pollution,  on  loss 
of  life  as  a  result  of  increased  traffic  accidents,  and  so 
forth. 

What  characterizes  all  these  effects,  Mishan  says, 
is  that  they  are  unintentional  and  not  subject  to  con- 
trol by  the  people  who  experience  them  (at  least  not 
unless  there  is  some  way  such  people  can  spend  mon- 
ey to  avoid  them)  (410).  Mishan  suggests  that  the 
number  of  external  effects  in  the  real  world  is  virtual- 
ly unlimited.  He  believes  society  is  obliged  to  limit 
them  for  the  economist. 

The  number  of  effects  that  can  be  internalized  into 
the  pricing  mechanism,  Mishan  says,  is  limited.  Ac- 
cording to  Mishan,  the  costs  of  spillovers  such  as 
traffic  noises,  pollution,  radioactive  waste,  and 
diseases  of  the  nerves,  heart,  and  stomach  caused  by 
high-tension  living  cannot  be  internalized.  Internal- 
ization would  require  that  the  potential  victim  of  the 
spillover  have  a  legal  property  right  to  some  measure 
of  quiet  or  clean  air,  freedom  from  tension,  and  so 
forth.  For  that  right  to  be  enforceable,  it  would  be 
necessary  to  delineate  a  territory  around  each  indi- 
vidual that  belongs  solely  to  that  person,  so  that  an 
intrusion  subject  to  legal  compensation  procedures 
could  be  identified.  Since  it  is  impossible  to  create  a 
market,  even  an  artificial  one,  that  would  make  pric- 
ing them  possible,  such  spillovers  must  ultimately  be 
evaluated  by  the  victims'  subjective  estimate. 

Deriving  his  argument  from  John  Stuart  Mill,  a 
19th  century  British  disciple  of  Bentham,  Mishan 
thinks  compensatory  sums  should  actually  be  paid  to 
victims  of  spillovers.  He  rejects  the  social  engineering 
approach  to  spillovers  which  seeks  to  formulate  tol- 
erance levels  for  society  (410): 

If  the  liberal  economist  rejects  social  engineering 
norms  such  as  "tolerance  level,"  it  is  not  merely  be- 
cause the  choice  of  such  a  level  for  society  is  necessar- 
ily arbitrary,  but  because  the  adoption  of  such  toler- 
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ance  norms  on  behalf  of  all  members  of  society  runs 

counter  to  the  doctrine  that  each  man  is  deemed  to  be 

the  best  judge  of  his  interests,  particularly  in  matters 

that  affect  him  intimately. 
A  good  portion  of  Mishan's  book  is  devoted  to  a  dis- 
cussion of  possible  compensation  and  legal  liability 
in  such  situations. 

Prest  and  Turvey  argue  that  CBA  is  not  relevant  to 
decisions  on  large  investments  because  such  large  in- 
vestments spill  over  so  much  that  they  may  end  up 
altering  the  whole  economy  (487).  The  example  they 
use  is  the  building  of  a  dam  in  an  underdeveloped 
country.  Prest  and  Turvey's  argument,  however, 
might  apply  equally  well  to  health  care. 

Grosse  recounts  that  HEWs  study  of  the  possibil- 
ities for  comprehensive  health  care  programs  for 
young  children  looked  good  enough  that  Congress 
became  very  interested  (265).  It  was  clear,  however, 
that  if  children  who  then  lacked  access  to  good  med- 
ical care  were  to  be  provided  with  conventional  pedi- 
atric services,  an  acute  shortage  of  doctors  would 
result.  Ways  had  to  be  found  to  use  medical  man- 
power more  efficiently.  Thus,  the  Social  Security 
Amendments  of  1967,  which  provided  for  early  case 
finding  and  treatment  of  birth  defects  and  chronic 
conditions  in  children,  also  provided  for  a  research 
and  demonstration  program  to  train  and  use  physi- 
cians' assistants. 

Training  physicians'  assistants  is  an  example  of  a 
side  effect,  because  such  training  was  not  counted  as 
a  cost  in  the  original  HEW  study.  Nor  was  it  antici- 
pated how  a  CBA  of  this  sort,  translated  by  law  into 
public  policy,  can  influence  the  practice  of  medicine. 
This  analysis  led  fairly  directly  to  an  increase  in  in- 
terest in  physicians'  assistants,  which  may  change  the 
whole  hierarchical  structure  of  medicine  in  the 
United  States.  It  is  not  clear  whether  such  a  develop- 
ment should  be  counted  as  a  cost  or  a  benefit,  or 
possibly  a  bit  of  each.  Ordinarily,  however,  a  mas- 
sive (usually  entirely  unforeseen)  ultimate  effect  such 
as  this  simply  does  not  figure  into  the  calculations. 

The  Problem  of  Pricing  the  Unpriceable 

Finally,  there  is  the  stubborn  problem  of  valuing 
intangibles,  or  pricing  the  unpriceable.  In  discussing 
intangible  costs  like  pain,  discomfort,  and  grief, 
Klarman  notes  that  one  difficulty  in  valuing  them  is 
that  they  accrue  partly  to  patients,  but  also  partly  to 
their  friends,  relatives,  and  to  society  (317).  One  way 
of  valuing  such  intangibles  is  to  ask  what  an  indi- 
vidual would  be  willing  to  pay  to  avoid  them.  But 
Klarman  believes  the  measurement  problems  here  are 
major.  Although  he  describes  a  number  of  different 
ways  such  intangibles  have  been  valued,  he  is  not 
very  enthusiastic  about  any.  Mishan  points  out  that 


Klarman  himself,  when  he  did  his  1965  calculations 
on  syphilis,  attributed  more  than  40  percent  of  the 
final  benefit  to  "reduction  of  stigma,"  which  was  val- 
ued (essentially  arbitrarily)  at  either  1  or  0.5  percent 
of  earnings  subsequent  to  the  discovery  of  syphilis 
(410). 

Parish,  along  with  most  other  commentators,  also 
discusses  the  difficulty  of  valuing  intangibles  (475).  It 
is  true,  says  Parish,  that  such  valuations  are  often  im- 
possible. Further,  he  notes  (475): 

.  .  .  ingenious,  indirect  means  are  often  unconvinc- 
ing and  tend  to  discredit  benefit-cost  analysis.  Also 
objectionable  is  the  practice  of  finely  calculating  the 
more  easily  quantifiable  elements  while  ignoring  or 
drawing  insufficient  attention  to  the  intangible  ones. 
Indeed,  this  may  be  a  more  heinous  event  than  to  at- 
tempt to  heroically  quantify  the  unquantifiable. 
Because  intangibles  are  not  sold  in  markets  where 
consumers  are  forced  to  reveal  their  preferences,  says 
Veatch,  the  willingness-to-pay  principle  is  difficult  to 
apply  to  them  (603). 

The  Fictional  "Facts" 

All  of  these  quite  genuine  measurement  difficulties 
mean  that  CBA  can  never  really  tell  the  unvarnished 
truth,  and  that  it  is  therefore  at  bottom  a  kind  of  sys- 
tematic misrepresentation  of  the  world  it  purports  to 
measure.  Self  goes  so  far  as  to  accuse  cost-benefit 
analysts  of  (556): 

.  .  .  unwittingly  or  not,  playing  a  confidence  trick 
with  the  symbols  of  monetary  exchange.  Of  course  the 
theoretical  welfare  economist  is  not  a  confidence 
trickster,  in  fact  he  is  often  high-minded,  but  he  is 
committed  to  the  discovery  of  some  ideal  ("optimum") 
set  of  economic  conditions  that  transcend  the  ordinar- 
ily market  economy,  with  the  aid  of  which  he  can 
measure  intrinsic  value. 
Many  commentators  believe  that  fact  saddles  econo- 
mists with  a  heavy  moral  responsibility,  and  speak  of 
it  in  exactly  such  terms.  Mishan,  for  instance,  ulti- 
mately characterizes  CBA  as  horse-and-rabbit  stew, 
the  rabbit  representing  costs  and  benefits  that  really 
can  be  quantified,  and  the  horse  representing  other 
considerations,  including  environmental  spillovers 
(410).  "No  matter  how  carefully  the  scientific  rabbit 
is  chosen,  the  flavor  of  the  resulting  stew  is  sure  to  be 
swamped  by  the  horse  flesh,"  he  concludes.  Econo- 
mists should  resist  the  temptation  to  ignore  the  horse. 

Implications  of  the  Bias  of  Science 
Toward  Rationality 

Both  distributional  considerations  and  the  truth  or 
falsity  of  the  assigned  values  in  CBA  are  value/ethi- 
cal problems  with  which  many  of  the  advocates  of 
such  analysis  are  all  too  familiar  and  with  which  they 
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are  constantly  struggling.  Another  value  issue  is 
more  subtle  and  does  not  appear  to  have  been  previ- 
ously identifed,  no  doubt  partly  because  it  underlies 
not  just  CBA,  nor  even  economics,  but  the  whole 
ethos  of  the  scientific  enterprise.  That  is  the  bias  of 
science  toward  knowledge  as  a  good  for  its  own  sake. 
This  bias  values  "rational"  decisions  (usually  defined 
as  decisions  made  on  the  basis  of  "fact")  over  "in- 
tuitive" ones  (usually  characterized  by  a  "gut  feel- 
ing"). 

As  Charles  Lindblom  and  David  Cohen  empha- 
size, policy  analysts  tend  to  believe  that  all  problems 
are  best  solved  by  gaining  more  information  (361). 
Over  and  over  again,  CBA  is  justified  not  on  grounds 
that  it  is  accurate  or  fair,  but  that  it  represents  in- 
formation, and  in  that  sense  is  and  will  always  be 
preferred  to  the  prior  state  of  ignorance.  Nobody 
puts  the  argument  more  succinctly  than  Prest  and 
Turvey,  who  note  that  CBA  forces  those  who  con- 
duct it  to  quantify  costs  and  benefits  insofar  as  possi- 
ble (487): 

.  .  .  rather  than  rest  content  with  vague  qualitative 
judgments  or  personal  hunches.  This  is  obviously  a 
good  thing  in  itself;  some  information  is  always  better 
than  none. 

Mancur  Olson  attributes  the  quest  for  rationality 
not  just  to  science  in  general,  but  to  economics  in 
particular  (469): 

.  .  .  Economic  (more  precisely  microeconomic)  the- 
ory is  in  a  fundamental  sense  more  nearly  a  theory  of 
rational  behavior  than  a  theory  of  material  goods. 
Boulding  grants  that  the  dollar  "is  a  dangerously  im- 
perfect measure  of  the  quality  of  human  life  and  hu- 
man values"  (67).  Nevertheless,  he  says  (67): 

...  it  is  a  useful  first  approximation,  and  in  these 
matters  of  evaluation  of  different  choices  it  is  extreme- 
ly useful  to  have  a  first  approximation  that  we  can 
then  modify.  Without  this,  indeed,  all  evaluation  is 
random  selection  by  wild  hunches. 
Layard  points  out  that  all  the  writers  in  his  edited 
volume  assume  that  knowledge  is  better  than  ignor- 
ance (349). 

Fein,  who  acknowledges  serious  problems  in  accu- 
rate valuation,  nevertheless  concludes  "these  prob- 
lems leave  us  no  worse  off  than  we  are  in  the  absence 
of  the  evaluation  effort"  (194).  In  fact,  he  says,  our 
awareness  of  the  problems  may  leave  us  better  off 
(194): 

.  .  .  the  cost-benefit  analysis  leads  to  a  greater  level 
of  understanding  of  the  deficiencies  in  our  measuring 
techniques,  of  the  vagueness  of  some  of  our  goals.  It 
does  not  make  us  ignorant  but  makes  us  aware  of  our 
ignorance.  It  forces  us  to  question  the  "conventional 
wisdom"  .  .  . 

Although  making  comparisons  between  people  or 
diseases  is  distasteful,  Fein  argues,  such  comparisons 


are  being  made  all  the  time  anyway.  It  is  preferable 
to  make  the  comparisons  explicitly  so  that  we  can  be 
aware  that  we  are  valuing  the  lives  of  airplane  pas- 
sengers, for  example,  more  highly  than  those  of  coal 
miners. 

Indeed,  several  writers  justify  CBA  by  arguing  it 
makes  us  face  squarely  the  moral  implications  of 
valuing  different  people's  lives  and  preferences  dif- 
ferently. Parish  observes  (in  connection  with  the  will- 
ingness-to-pay principle  and  the  difficulty  of  valuing 
intangibles  not  sold  in  markets)  that  even  to  point 
out  these  difficulties  is  useful  because  it  reveals  an 
area  of  ignorance  (475). 

Mishan  views  the  economists  doing  CBA  as  having 
a  much  more  active  role  than  that  of  a  simple  techni- 
cian (410).  To  him,  the  cost-benefit  analyst  is  a  moral 
agent  with  an  obligation  to  incorporate  some  of  the 
deficiencies  revealed  by  the  analysis  into  policy  for- 
mulation. Even  when  analysts  cannot  bring  some  im- 
portant but  unquantifiable  item  into  the  calculus, 
they  can  at  least  clearly  reveal  an  area  of  ignorance. 
They  can  also  help  valuation  by  providing  a  physical 
description  of  the  spillovers  and  their  significance 
and  offering  a  guess  or  range  of  guesses  about  the 
value  of  the  damage.  Analysts  can  also  provide  con- 
tingency calculations  and  estimate  the  critical  mag- 
nitude for  the  spillovers  that  will  just  offset  a  proj- 
ect's benefits.  They  can  even  use  questionnaires,  says 
Mishan  (410): 

Surveys  based  on  the  questionnaire  method  may  be 
suspect  for  a  number  of  reasons,  but  they  are  some- 
times better  than  guesswork,  and  assuredly  better 
than  no  information  at  all. 

Again  and  again,  in  discussion  after  discussion, 
commentators  state  the  better-than-nothing  argu- 
ment. Despite  these  expressions  of  faith,  the  question 
remains  whether  revealing  our  ignorance  will,  in 
fact,  force  us  to  face  and  remedy  it.  Grosse  observes 
that  the  HEW  study  of  maternal  and  child  health  pro- 
grams was  a  difficult  one  to  do  because  hard  infor- 
mation on  the  state  of  children's  health  was  difficult 
to  obtain  (265): 

Surprisingly,  estimates  of  improvement  in  general 
health  attributable  to  medical  care  are  almost  nonex- 
istent. It  is  not  easy  to  demonstrate  statistically  that 
children  who  see  doctors  regularly  are  healthier  than 
children  who  do  not. 
This  statement  is  an  indication  of  what  we  can  prob- 
ably take  to  be  Grosse  s  belief  that  visits  to  the  doctor 
do  indeed  improve  health;  he  is  frustrated  only  by  his 
inability  to  demonstrate  that  statistically.  The  alter- 
native view,  both  plausible  and  increasingly  fashion- 
able, is  that  visits  to  the  doctor  do  not  contribute  ap- 
preciably to  the  general  health  of  children.  That 
possibility  appears  not  to  have  occurred  to  Grosse, 
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though  entertaining  it  might  certainly  give  policy- 
makers quite  a  different  view  of  what  kinds  of  prior- 
ities to  accord  to  health  care. 

One  of  the  most  extensive  criticisms  of  this  ration- 
alism-oriented mind  set  has  been  lodged  by  Veatch. 
He  argues  that  systems  analysis,  rationalized  cen- 
tralized planning,  and  quantification  are  not  value- 
neutral,  but  are  instead  attractive  to  people  who  have 
certain  values  (603).  Veatch  criticizes  systematic, 
data-based  analysis  of  policy  problems  on  two 
grounds.  One  is  practical —  he  has  doubts  about  its 
outcome.  So  many  things  in  practice  turn  out  to  be 
unquantifiable,  Veatch  says,  that  it  may  be  better  not 
to  pursue  such  analysis  at  all  because  the  possibility 
of  error  may  be  great.  Furthermore,  Veatch  says,  sys- 
tematic, data-based  analyses  of  policy  problems  are 
contrary  to  a  sense  of  human  freedom  or  spontaneity 
and  are  overly  rationalistic  (603): 

It  is  sobering  to  realize  that  basic  lifestyle  choice  is 
at  stake  here.  One  cannot  systematically  or  rationally 
prove  that  systematic,  rational  policy  analysis  is  a 
preferable  style  of  living  in  a  social  community. 
For  Lindblom  and  Cohen,  the  problem  is  not  style 
but  substance  (361).  They  believe  that  more  knowl- 
edge will  be  of  little  use  in  many  situations  and  that 
many  problems  are  best  solved  "interactively,"  that 
is,  by  means  of  negotiation  or  the  political  process. 
Even  when  knowledge  is  of  value,  it  is  usually  only 
one  element  in  a  more  complex  political  and  human 
situation.  Moreover,  they  argue,  interactive  problem 
solving  can  be  just  as  rational  in  its  methodology  as 
the  "scientific"  approach. 

Economics,  like  the  rest  of  science,  however,  is  tied 
irrevocably  to  the  latter.  As  Boulding  observes  (67): 
The  fundamental  principle  that  we  should  count  all 
costs,  whether  easily  countable  or  not,  and  evaluate 
all  rewards,  however  hard  they  are  to  evaluate,  is  one 
which  emerges  squarely  out  of  economics  and  which 
is  at  least  a  preliminary  guideline  in  the  formation  of 
the  moral  judgment,  in  what  might  be  called  the  "eco- 
nomic ethic." 

Boulding  freely  acknowledges  that  some  things  sim- 
ply cannot  be  measured,  particularly  what  he  calls 
the  "heroic  ethic"  (bravery  in  the  military,  religious 
feelings,  and  so  forth)  (67): 

The  attack  on  economics  is  an  attack  on  calculated- 
ness,  and  the  very  fact  that  we  think  of  calculating  as 
cold  suggests  how  exposed  economists  are  to  romantic 
and  heroic  criticism. 
The  problem  is  to  retain  both  heroic  and  economic 
elements  in  our  institutions  and  in  our  decisionmak- 
ing, but  in  proper  balance  (67): 

Economic  man  is  a  clod,  heroic  man  is  a  fool,  but 
somewhere  between  the  clod  and  the  fool,  human 
man,  if  the  expression  may  be  pardoned,  steers  his  tot- 
tering way. 


Though  most  instinctively  prefer  even  a  flawed 
systematic  analysis  to  none,  it  is  well  to  keep  Fein's 
warning  in  mind  (194): 

While  it  is  better  to  know  something  than  to  know 

nothing  ...  we  dare  not  minimize  the  danger  that  in 

knowing  something  we  may  behave  as  if  we  know 

everything. 

Implications  off  the  Conflict  Between 
Freedom  and  Paternalism 

The  conflict  between  freedom  and  paternalism  is 
one  not  much  discussed  in  the  literature  on  CBA.  But 
individual  freedom  is  an  issue  particularly  important 
in  the  United  States,  and  paternalism  one  particularly 
important  in  medicine.  The  conflict  between  freedom 
and  paternalism  is  thus  a  perennial  one  in  U.S.  health 
care.  Clearly,  CBA,  with  its  quantitative  methods 
and  scientific  respectability,  can  force  choices. 

Nevertheless,  it  is  anything  but  clear  that  CBA  has 
done  this.  One  of  the  early  analyses  performed  at 
HEW  and  described  by  Grosse  (265),  for  example,  ex- 
amined the  value  of  fluoridation  of  drinking  water  to 
help  reduce  cavities.  Grosse  says  that  on  paper  (265): 
.  .  .  fluoridation  looks  like  a  very  attractive  pro- 
gram. It  was  so  attractive  that  one  can  assume  that  a 
program  as  cheap  as  this  is  not  bring  inhibited  by  a 
lack  of  financial  support  by  the  Federal  Government; 
there  are  other  factors  at  work. 
He  does  not  specify  the  nature  of  those  other  factors, 
but  most  of  us  know  they  are  political.  There  is 
strong  local  opposition  to  fluoridation  of  water, 
often  on  grounds  that  it  is  simply  wrong  to  force 
everybody  in  a  community  to  undergo  a  medical 
treatment  with  no  way  of  opting  out. 

Public  policy,  particularly  health  policy,  has  often 
been  criticized  as  paternalistic.  The  Government  is 
often  accused  of  meddling  in  people's  lives,  trying  to 
get  them  to  behave  in  certain  ways  for  their  own 
good.  CBAs  are,  in  a  way,  a  response  to  that  criti- 
cism. Today,  health  programs  are  justified  less  on 
grounds  that  the  Government  ought  to  be  doing  good 
things  for  its  citizens  than  in  terms  of  the  tax  savings 
and  other  economic  benefits  that  will  result.  Thus, 
screening  newborns  for  PKU  is  no  longer  (or  rarely) 
supported  on  grounds  that  families  ought  to  be 
spared  the  miseries  of  having  to  cope  with  a  retarded 
child  who  need  not  have  been  so  retarded,  but  rather 
on  grounds  that  the  Government  will  save  money  on 
institutional  costs  for  the  retarded  by  underwriting 
such  a  program. 

It  is  probable,  however,  that  in  a  way  CBA  func- 
tions partly  as  a  mask  for  the  old  paternalism.  The 
extensive  analyses  of  genetic  screening,  for  instance, 
offer  a  respectably  scientific  and  economic  justifica- 
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tion  for  continuing  programs  whose  origins  are  actu- 
ally traceable  to  a  combination  of  political  and  hu- 
mane impulses.  This  illustrates  that  the  important 
problem  here  may  not  be  the  paternalism  of  econo- 
mists, but  that  of  the  policymaker.  The  Government 
policymaker  orders  and  pays  for  the  analysis.  Be- 
cause of  this,  the  policymaker  will  often  have  much 
to  say  about  what  goes  into  the  analysis,  for  in- 
stance, concerning  goals  and  outcomes.  Further- 
more, the  policymaker  is  also  in  a  position  to  apply 
the  analysis  to  the  making  of  social  decisions.  This 
position  not  only  invites  paternalism,  it  almost 
demands  it,  since  in  some  sense  policymakers  are  by 
definition  in  a  somewhat  paternal  role  with  respect  to 
the  people  for  whom  they  make  policy. 

Of  course,  a  CBA  can  also  be  influenced  by  the 
degree  of  paternalism  subscribed  to  by  economists. 
For  example,  many  analyses  employ  shadow  pricing 
in  which,  for  a  variety  of  reasons,  the  value  of  some 
item  in  the  analysis  is  set  at  something  other  than  its 
current  market  value.  As  Layard  points  out,  highly 
paternalistic  economists  may  very  well  substitute 
their  own  valuation  for  that  revealed  by  answers  to 
survey  questions  if  they  believe  that  the  public's 
valuations  may  not  be  accurate — particularly  when 
valuing  future  possibilities  (349). 

Conclusions 

The  issues  of  equity,  efficiency,  freedom,  pater- 
nalism, the  value  of  knowledge,  and  rationality — by 
truth-telling  and  deception — are  irretrievably  im- 
bedded in  the  techniques  of  CEA  and  CBA.  There  is 
no  way  to  change  the  techniques  to  eliminate  those 
values  and  their  inevitable  conflicts.  Certain  tech- 
nical adjustments  (e.g.,  putting  the  services  of  house- 
wives into  dollar  terms)  have  been  made  in  the  past, 
and  these  have  improved  the  situation  somewhat. 
Similar  adjustments  will  continue  to  be  made  in  the 
future,  but  essentially  they  will  be  small  adjustments 
that  will  have  no  effect  on  the  fact  that  the  issues  de- 
scribed are  simply  inherent  in  the  techniques  them- 
selves or  in  their  translation  into  policy. 

Should  we  not  use  cost-benefit  techniques  in  forg- 
ing health  policy?  No,  discarding  CBA  is  not  the 
answer.  In  an  important  sense,  the  defenders  of  the 
technique  are  correct  when  they  argue  that  policy 
decisions  in  health  care  are  being  made  daily  on 
shaky  grounds  anyway.  CBA  is  at  least  an  attempt, 
however  imperfect,  to  ground  those  decisions  in  real 
needs  and  real  possibilities. 

The  problem  is  not  that  CBA  lacks  objectivity  and 
is  not  value  free,  but  rather  that  objectivity  and  value 
freedom  are  unjustifiably  attributed  to  it.  To  their 
credit,  most  theorists  of  CBA  cited  here  are  quite 


aware  that  their  techniques  are  anything  but  objec- 
tive and  value  free.  The  problem  comes  with  lesser 
practitioners  who  see  themselves  simply  as  techni- 
cians or — and  this  may  be  the  real  problem — with 
policymakers  who  take  the  scientific  aura  of  CBA  for 
truth,  failing  to  realize  CBA's  limitations. 

Many,  discussions  of  CBA  have  paid  little  atten- 
tion to  policymakers.  Yet,  virtually  all  analyses  are 
undertaken  at  the  behest  of  policymakers,  and  the 
methods  employed  in  a  specific  analysis  may  depend 
on  their  goals  and  values.  Weinstein  and  Stason 
point  out,  for  instance,  that  a  health  maintenance 
organization  may  care  about  total  costs,  whereas  a 
group  practice  may  care  chiefly  about  nonreimburs- 
able costs;  such  differences  have  obvious  and  impor- 
tant implications  for  society's  efforts  to  contain  costs 
(627). 

To  understand  the  pros  and  cons  of  CBA  studies, 
one  must  also  understand  the  policy  level  on  which 
they  are  commissioned  and  applied.  Who  is  the  pol- 
icymaker? The  Secretary  of  DHHS,  the  Senate  Sub- 
committee on  Health  and  Scientific  Research,  the 
State  health  department,  the  health  systems  agency, 
the  hospital  administrator,  the  school  nurse,  the  pri- 
vate physician?  Each  of  these  policymakers  may  very 
well  use  CBA  to  make  policy.  The  decision  may  be 
different  in  every  case,  however,  because  individual 
policymakers,  like  economists,  will  have  personal 
values,  institutional  biases,  and  a  somewhat  different 
constituency  from  others'. 

Harold  Green  notes  that  optimism  or  pessimism 
can  sometimes  be  "a  function  of  statutory  mandate 
or  agency  mission"  (258).  The  Food  and  Drug  Ad- 
ministration, for  instance,  automatically  regards  any 
food  additive  as  hazardous  until  it  has  been  demon- 
strated to  be  safe.  The  importance  of  the  policy- 
maker's perspective  is  illustrated,  as  well,  by  the 
unsettled  question  of  whether  a  physician  ought  to 
depart  from  concern  for  the  patient's  welfare  and 
begin  to  incorporate  social  values  into  the  decision 
to,  for  example,  order  an  additional  test  that  has  only 
very  small  chance  of  giving  additional  information. 

For  some  analysts,  the  question  of  who  the  policy- 
maker is  does  not  seem  to  be  very  important.  If  as  an 
analyst  you  follow  Mishan's  recommendations  (410), 
for  instance,  you  will  behave  quite  explicitly  as  a 
moral  agent,  with  a  duty  to  point  up  all  the  relevant 
effects  of  a  program  that  do  not  enter  the  cost-benefit 
computation  no  matter  who  your  employer  is.  Mi- 
shan  also  argues  that  if  economists  have  reason  to  be- 
lieve that  a  project  will  be  unambiguously  regressive, 
it  is  their  duty  to  mention  this;  for  Mishan,  econo- 
mists should  be  just  as  committed  to  equity  as  to  effi- 
ciency. That  commitment  should  obtain,  he  implies, 
regardless  of  the  level  on  which  policy  is  being  made. 
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It  is  not  clear  whether  Mishan's  view  is  widely 
shared  by  economists,  or  whether  instead  most  econ- 
omists see  themselves  as  simple  hired  hands.  At  a 
minimum,  though,  it  seems  sensible  to  assert  that 
there  are  some  kinds  of  information  that  ought  to  be 
made  explicit  as  part  of  a  CBA.  These  include  clear 
indications  of  the  data  sources  on  valuing  lives,  what 
kinds  of  equity  considerations  spring  from  the  use  of 
particular  data  (e.g.,  whether  whites  are  valued  more 
highly  than  blacks),  and  how  and  why  the  discount 
rate  was  selected. 

Conventional  assumptions,  disclaimers,  and  cau- 
tions also  ought  to  be  an  explicit  part  of  each  CBA. 
Greater  use  of  sensitivity  analysis  would  probably  be 
illuminating.  In  addition,  people  need  to  be  made 
more  aware  of  the  limitations  of  CBA.  That  probably 
includes  people  within  the  economics  profession 
itself,  certainly  includes  policymakers  who  use  CBA, 
and  may  even  include  the  general  public. 

CBA  should  certainly  never  be  used  as  the  sole  ba- 
sis for  major  health  policy  decision.  Such  use,  how- 
ever, may  not  be  as  much  of  a  problem  as  has  often 
been  charged.  Grosse  for  instance,  says  the  HEW 
analyses  were  used  to  give  the  Department  a  "feel  for 
what  were  relatively  high-  and  low-priority  pro- 
grams, and  then  to  feed  these  insights  into  the  deci- 
sionmaking process,  which  also  considers  other 
viewpoints,  the  existing  commitments,  the  political 
situation,  the  rate  of  spending,  and  the  ability  to  get 
people  moving  on  programs,  and  so  on"  (265).  In 
Grosse's  view,  CBA  is  clearly  a  useful  tool,  but  only 
one  of  many. 

Swint  and  Nelson  apparently  share  this  view,  ob- 
serving (593): 

Many  of  the  criticisms  of  (cost-benefit  analysis) 
come  from  the  people  who  have  (perhaps  inadvertent- 
ly) set  up  a  "straw  man"  by  claiming  it  cannot  provide 
the  sole  decision-making  criterion.  The  point  is  that 
(cost-benefit  analysis)  is  not  intended  to  provide  the 
sole  decision-making  criterion;  that  has  only  been 
incorrectly  imputed  to  it  by  non-practitioners.  What  it 
does  do  is  provide  economic  information  that  must  be 
combined  with  distributional,  sociopolitical,  humani- 
tarian and  other  information  by  the  decision  maker 
(vis-a-vis  the  analyst)  for  net  evaluation.  We  see  no 
point  in  depriving  decisionmakers  of  one  piece  (the 
economic)  of  the  several  sets  of  information  needed  to 
make  rational  decisions. 

For  Prest  and  Turvey,  who  were  writing,  after  all, 
in  the  comparatively  early  days  of  1965,  CBA  is 
"only  a  technique  for  making  decisions  within  a 
framework  which  has  to  be  decided  upon  in  advance 
and  which  involves  a  wide  range  of  considerations, 
many  of  them  of  a  political  or  social  character"  (487). 
They  argue  that  the  case  for  using  CBA  is  strength- 
ened, not  weakened,  if  its  limitations  are  openly 
recognized  and  emphasized.  CBA  may  then  function 


as  a  "sensible  antidote  to  the  wilder  excesses  of  par- 
ticular lobbies."  CBA  may  also  cause  questions  to  be 
asked  that  would  not  otherwise  have  been  raised. 
Prest  and  Turvey  also  agree  with  Grosse  that  even  if 
CBA  cannot  always  give  the  right  answers,  it  can 
sometimes  play  the  purely  negative  role  of  screening 
projects  and  rejecting  those  that  are  clearly  less 
promising  (487,265). 

On  the  other  hand,  one  should  certainly  not  under- 
estimate the  power  of  politics.  Klarman  points  out 
that  politics  is  one  of  the  foremost  barriers  to  system- 
atic analysis  (265).  CBA  implies  a  delineation  of 
goals  and  an  articulation  of  values,  whereas  the  polit- 
ical process,  he  says,  may  require  the  blurring  of  dif- 
ferences and  conflicts  in  order  to  facilitate  coalition 
building  so  that  particular  ends  can  be  achieved.  Ac- 
cording to  Martin  Rein,  because  of  that  requirement 
of  politics,  it  is  necessary  to  put  planning  beyond  the 
reach  of  shortsighted  political  considerations: 
".  .  .  at  some  stage,  and  to  some  degree,  decisions 
must  be  taken  out  of  the  political  context  and  handed 
over  to  the  social  scientist  for  analysis"  (501).  Real 
exploration  and  analysis  of  alternative  choices,  he 
suggests,  can  occur  only  in  the  absence  of  short-term 
political  pressures. 

How  decisions  can  be  taken  out  of  the  political 
context  for  analysis  is  not  clear.  Nor  is  it  entirely 
clear  that  they  should  be.  Green  argues  that  some- 
times nonscientific  negotiation  and  compromise  may 
be  the  best  course  for  policymakers  (258,361).  He 
asks  (258): 

Is  it  not  more  important  to  resolve  controversy  in 
the  optimum  manner  than  to  produce  an  objectively 
correct  result?  If  so,  is  it  not  possible  that  optimum 
resolution  of  controversy  may  require  acceptance  as 
true  of  something  that  we  know  is  untrue? 
Even  some  of  CBA's  best-known  and  most  sensi- 
tive practitioners  acknowledge  that,  at  times,  the 
political  process  may  respond  better  to  our  needs 
than  even  the  most  rational  and  careful  CBA.  Rashi 
Fein  believes  in  CBA.  He  thinks  that  policymakers 
need  to  be  reminded  that  there  are  economic  returns 
to  health  programs,  that  good  health  can  be  sup- 
ported on  investment  grounds,  and  that  poor  health 
costs  a  lot.  On  the  other  hand,  Fein  comments  (194): 
At  present,  in  the  United  States,  beset  as  we  are  by 
divisions  and  by  tensions,  distributional  considera- 
tions lie  at  the  heart  of  many  of  our  problems.  The 
healing  of  social  wounds  (not  an  unimportant  objec- 
tive even  if  its  benefits  cannot  be  quantified  in  mone- 
tary returns)  may,  today,  be  more  readily  accom- 
plished by  providing  the  services  that  people  believe 
to  be  important  than  by  providing  that  which  the  ana- 
lyst has  tentatively  determined  is  most  beneficial.  The 
healing  of  social  wounds,  is,  at  this  moment,  I  believe 
more  vital  than  the  healing  of  disease. 
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Most  of  the  existing  statutory  authorities  that 
guide  Federal  health-care-related  agencies  allow  the 
agencies  to  use  or  support  CEA/CBA.  Recently, 
however,  two  legislative  initiatives — Public  Law 
95-623  establishing  the  National  Center  for  Health 
Care  Technology  (NCHCT)  and  the  National  Health 
Planning  and  Resources  Development  Amendments 
to  Public  Law  93-641— placed  special  emphasis  on 
the  use  and  support  of  CEA  by  NCHCT  and  by 
health  systems  agencies  (HSAs).  Although  these  laws 
are  not  mandates,  they  are  explicit  suggestions  to  do 
CEA  or  CBA  studies.  The  language  of  Public  Law 
95-623  calls  for  NCHCT  to  "give  appropriate  empha- 
sis" to  cost  effectiveness  (among  a  number  of  other 
criteria)  while  conducting  and  supporting  research  on 
health  care  technology.  Similarly,  the  health  plan- 
ning amendments  state  that  cost  effectiveness  should 
be  one  of  the  criteria  that  HSAs  use  while  carrying 
out  their  mandate  to  review  the  appropriateness  of 
existing  facilities  (see  ch.  7  for  a  detailed  discussion). 
These  two  laws  are  as  close  as  Congress  has  come  to 
requiring  a  health-care-related  agency  to  perform  or 
support  CEA  or  CBA  studies  as  part  of  its  mission. 

At  the  other  end  of  the  legislative  spectrum  from 
these  two  laws  are  the  statutes  that  authorize  and 
guide  the  efforts  of  the  Food  and  Drug  Administra- 
tion (FDA).  FDA  has  traditionally  interpreted  its 
mandate  as  excluding  or  strongly  discouraging  the 
formal  evaluation  and  incorporation  of  economic 
costs  and  social  benefits  in  its  regulatory  proceedings 
(127a).  Although,  FDA  has  done  a  small  number  of 
costs  and  benefits  type  comparisons  of  proposed  in- 
itiatives, those  analyses  were  conducted  in  response 
to  executive  orders  calling  for:  Inflationary  Impact 
Statements  (E.O.  11821,  1974),  Economic  Impact 
Statements  (E.O.  11949,  1977),  and  Improving  Gov- 
ernment Regulations  (E.O.  12044,  1978).  Those 
orders  applied  to  executive  agencies  that  proposed 
major  regulations,  i.e.,  regulations  having  an  annual 
impact  on  the  economy  of  $100  million  or  more  or 
causing  a  major  increase  in  costs  or  prices  for  individ- 
ual industries,  levels  of  government,  or  geographic 
regions.  Any  agency  that  proposed  a  regulation  fall- 
ing into  one  of  these  categories  was  required  to  ex- 
amine the  costs  and  benefits  of  the  proposed  action. 

One  section  of  the  food  additive  amendments  to 
FDA's  mandate,  usually  referred  to  as  the  "Delaney 
clause,"  (The  Food,  Drug,  and  Cosmetic  Act,  21 
U.S.C.,  sec.  409c3a,  1976)  explicitly  prohibits  the 
balancing  of  risks  and  benefits  in  a  proposed  regula- 
tion of  a  food  additive  that  has  been  shown  to  be  car- 
cinogenic (127a).  The  inflexible  language  of  this 
amendment  has  stimulated  a  number  of  legislative 


proposals  to  repeal  or  alter  the  "Delaney  clause," 
and/or  alter  FDA's  authorizing  legislation  to  allow  or 
require  the  balancing  of  risks  and  benefits  in  pro- 
posed regulatory  or  rulemaking  initiatives.1  The 
most  prominent  of  the  initiatives  is  the  bill  intro- 
duced in  the  96th  Congress  as  the  "Drug  Regulation 
Reform  Act  of  1979." 

The  legal  status  of  CEA  and  CBA  in  the  decision- 
making processes  of  other  agencies  of  the  Federal 
Government  is  in  a  much  greater  state  of  conflict, 
confusion,  and  flux.  In  the  past  10  years,  numerous 
court  rulings  have  attempted  to  define  and  clarify  the 
role  of  CEA  and  CBA  in  the  decisionmaking  proc- 
esses of  regulatory  agencies.  Increasing  congressional 
attention  is  being  focused  on  the  potential  usefulness 
of  CEA  and  CBA  techniques,  especially  as  they  per- 
tain to  the  decisionmaking  efforts  of  several  regula- 
tory agencies.  Over  65  bills  were  introduced  in  the 
last  two  Congresses  that  would  require  various  agen- 
cies to  incorporate  CEA,  CBA,  or  risk-benefit  tech- 
niques into  their  formal  decisionmaking  procedures 
(600).  For  many  agencies,  the  legislation  would  sim- 
ply formalize  a  CEA/CBA  process  that  is  now  infor- 
mal. In  a  number  of  other  agencies,  a  formal  or  infor- 
mal process  of  using  CEA/CBA  does  not  exist.  In  any 
case,  many  of  the  proposed  changes  would  make  ex- 
plicit that  which  past  legislation  has  left  vague  and 
open  to  judicial  and  agency  interpretation. 

In  the  past  decade,  a  number  of  health,  safety,  and 
environmental  laws  have  added  to  the  Federal  Gov- 
ernment's involvement  in  the  regulation  of  highly 
controversial  areas  of  society  and  business.2  In  many 
instances,  the  specific  purpose  and  intended  out- 
comes of  the  legislation  were  inadequately  defined 
after  the  general  goals  were  established.  In  addition, 
the  types  of  procedures  allowed  or  required  to  initi- 
ate agency  action,  or  the  level  of  "evidence"  required 
to  support  a  given  initiative,  were  left  vague  in  the 
authorizing  legislation.  In  many  cases,  these  things 
were  left  vague  with  good  reason.  The  uncertainties 


'Most  of  the  reaction  was  stimulated  by  the  use  of  the  "Delaney  clause"  to 
initiate  a  ban  of  saccharin,  an  artificial  sweetener  used  extensively  in  this 
country.  For  a  detailed  analysis  of  the  technical  issues  involved  in  the  pro- 
posed saccharin  regulation,  see  Cancer  Jesting  Technology  and  Saccharin 
(466),  prepared  by  OTA. 

included  among  these  laws  are  (127a):  1)  Safety  and  Health  Act  (Public 
Law  91-596);  2)  Consumer  Product  Safety  Act  (Public  Law  92-573);  3) 
Amendments  to  the  Pesticide  Act  (Public  Law  92-516,  Public  Law  94-140, 
Public  Law  95-396);  4)  Solid  Waste  Disposal  Act  (Public  Law  94-580);  5) 
Toxic  Control  Act  (Public  Law  94-469);  6)  Medical  Device  Amendments 
(Public  Law  94-295);  7)  Safe  Drinking  Water  Act  of  1977  (Public  Law 
95-190);  8)  Clean  Air  Act  Amendments  of  197?  (Public  Law  91-604,  Public 
Law  95-95).  Perhaps  the  most  well  known  is  the  National  Environmental 
Policy  Act  of  1969—  NEPA  (U.S.C.  sec.  4321  et  seq.,  1970).  It  has  been  re- 
sponsible for  a  significant  number  of  legal  challenges  regarding  the  deci- 
sionmaking methods  of  a  host  of  Federal  agencies. 
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involved  in  the  issues,  the  imprecise  nature  of  the 
best  available  data,  and  many  other  factors  required 
that  the  authorizing  statutes  remain  flexible. 

The  broadly  worded  statutory  language,  however, 
has  left  the  door  open  for  procedural  challenges,  dif- 
fering interpretations  of  the  laws,  and  litigation.  The 
practical  result  of  this  problem  is  that  the  courts  have 
been  repeatedly  forced  to  clarify  and  define  what  is 
required  of  the  agencies  in  establishing  a  rule  or  reg- 
ulation. Numerous  court  cases  have  challenged  the 
decisionmaking  procedures  of  the  Environmental 
Protection  Agency,  the  Consumer  Product  Safety 
Commission  (CPSC),  the  Nuclear  Regulatory  Com- 
mission, the  Occupational  Safety  and  Health  Admin- 
istration (OSHA),  and  others.3  One  of  the  results  of 
this  litigation  is  an  increasing  pressure  on  the  agen- 
cies to  include  more  formal  and  explicit  costs  v.  bene- 
fits comparisons  in  their  decisionmaking  and  evi- 
dence-gathering procedures. 

Neither  the  courts  nor  Congress  are  totally  unfa- 
miliar with  CEA  or  CBA  techniques.  The  Federal 
courts  have  been  concerned  with  agency  actions 
using  costs  and  benefits  since  the  early  1940's  (526). 
Congress  has  required  the  Corps  of  Engineers  to  jus- 
tify water  resource  projects  with  cost-benefit  calcula- 
tions since  the  late  1930's.  Recent  years,  especially 
the  last  10,  however,  have  witnessed  an  increasing 
awareness  of  the  use  of  CEA/CBA  in  the  regulatory 
decisionmaking  process.  Enactment  of  the  National 
Environmental  Policy  Act  (NEPA)  in  1969  and  the 
issuance  of  the  executive  orders  mentioned  above 
marked  the  beginning  of  significant  judicial  involve- 
ment in  the  area  of  procedural  and  substantive  re- 
view of  CEA/CBA  use  in  the  regulatory  process. 

Despite  a  decade  of  litigation,  conflict,  and  debate, 
the  role  of  formal  CEA/CBA  in  the  regulatory  proc- 
ess still  remains  ill-defined  and  unclear  in  many,  if 
not  most,  decisionmaking  situations.  The  courts 
have  had,  and  continue  to  have,  a  difficult  time  in 
producing  a  coherent  body  of  law  that  establishes 
uniform  standards  of  judicial  review  in  this  area  (this 
is  especially  true  for  NEPA-related  issues).  They  have 
not  been  able  to  reconcile  the  conflicting  standards 
that  have  been  placed  into  various  statutes  that  in- 
directly require  many  agencies  to  balance  the  social 
costs  and  benefits  of  their  regulatory  initiatives  (526). 
The  courts  review  agency  initiatives  with  three  basic 
criteria  in  mind:  1)  Did  the  agency  conform  to  the 
procedures  set  out  in  the  law?  (the  procedural  test); 
2)  Was  the  cost-benefit  comparison  arbitrary  or  ca- 

This  appendix  does  not  attempt  to  provide  a  comprehensive  discussion 
of  the  development  and  interpretation  of  the  case  law  that  guides  this  area 
of  regulatory  behavior.  At  best,  an  attempt  is  made  to  touch  on  the  recent 
judicial  highlights  that  have  built  upon  earlier  rulings.  For  a  more  complete 
analysis  of  the  courts'  involvement  in  this  area,  see  references  41,  42,  and 
526. 


pricious?  (a  substantive  and  due  process  test);  and  3) 
What  is  the  "substantial  evidence  on  the  record  as  a 
whole?"  (also  a  substantive  test).  Putting  these  cri- 
teria into  practice  has  not  always  been  easy  or  suc- 
cessful. The  limitations  of  the  methods  of  CEA/CBA, 
the  vagueness  of  the  legislative  language  and  intent, 
the  differing  interpretations  of  the  level  of  sophistica- 
tion or  rigor  required  in  the  cost-benefit  compari- 
sons, and  the  reluctance  of  the  courts  to  substitute 
their  judgments  for  agency  expertise  have  contrib- 
uted to  the  continuing  confusion  that  surrounds  the 
use  of  formal  CEA/CBA  techniques  in  the  decision- 
making process. 

A  number  of  recent  and  pending  Federal  District 
Court  rulings  may  be  able  to  give  further  indications 
of  the  direction  in  which  the  courts  are  moving  with 
respect  to  requiring  CEA  or  CBA  techniques  in  the 
decisionmaking  process.  One  should  keep  in  mind, 
however,  that  the  courts'  opinions  may  be  specific  to 
an  agency,  may  be  subject  to  later  reinterpretation  by 
a  higher  court,  or  may  be  applicable  only  to  the  spe- 
cific statutory  wording  on  which  the  rulings  turn. 
OTA  is  not  suggesting  that  these  opinions  will  have 
significant  impact  on  the  health-related  agencies  ex- 
amined in  this  assessment.  As  judicial  signals  that 
may  very  well  broadcast  to  a  number  of  executive 
agencies  the  extent  to  which  formal  CEA/CBA  tech- 
niques might  be  used  or  required  in  their  decision- 
making procedures,  however,  these  opinions  are 
worthy  of  note. 

The  litigation  many  observers  may  be  watching 
most  closely  is  the  "benzene  case"  now  before  the 
Supreme  Court.4  That  case  involves  a  number  of  is- 
sues, but  the  primary  focus  is  on  two  questions:  1)  Is 
OSHA  required  to  use  a  costs  v.  benefits  comparison 
to  support  its  proposed  standards?  and,  2)  If  so,  is  the 
analysis  OSHA  claims  to  have  performed  adequate 
to  support  its  decision?  A  lower  court  answered  yes 
to  the  first  and  no  to  the  second,  and  overturned 
OSHA's  proposed  benzene  standards  for  the  work- 
place (20).  The  lower  court  held,  in  part,  that  OSHA 
had  failed  to  properly  or  adequately  compare  the 
benefits  expected  from  the  proposed  regulation  to  the 
anticipated  costs  of  compliance  to  determine  if  a 
"reasonable  relationship"  existed.5 


'Industrial  Union  Department,  AFL-CIO  v.  American  Petroleum,  et  al. 
(No.  78-911)  and  Marshall  v.  American  Petroleum  Institute,  et  al.  (No. 
78-1036). 

The  language  in  OSHA's  mandate  (Occupational  Safety  and  Health  Act, 
29  U.S.C.  sec.  652  et  seq.  (1975))  requires  that  OSHA  has  the  obligation  to 
enact  only  standards  that  are  "reasonably  necessary"  or  appropriate  to  pro- 
vide safe  or  healthful  workplaces  and  that  standards  dealing  with  toxic  ma- 
terials be  "feasible."  OSHA  must  determine  whether  benefits  expected  from 
a  standard  bear  a  reasonable  relationship  to  costs  imposed  by  the  standard. 
The  Circuit  Court  rulings  in  the  CPSC  and  OSHA  cases  hinge  on  these 
phrases;  "reasonably  necessary,"  "feasible,"  and  whether  benefits  expected 
bear  a  "reasonable  relationship"  to  the  costs  expected. 
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Circuit  Court  rulings  involving  CPSC  were  cited 
by  the  court  in  the  benzene  case  to  illustrate  the  evi- 
dence requirements  that  the  court  felt  are  necessary 
to  determine  if  a  proposed  initiative  is  "reasonably 
necessary"  when  the  proposed  standard's  benefits  are 
balanced  against  the  costs,  or  burden,  of  compliance 
with  the  standard.6  The  Supreme  Court  will  deter- 

"The  rulings  cited  in  the  OSHA  opinion  as  guiding  the  determinations 
were: 

1.  Aqua  Slide  'N'  Dive  v.  Consumer  Product  Safety  Commission,  569  F.  2d. 
832  (5th  Cir.  1978).  and 

2.  D.  D.  Bean  &  Sons  v.  Consumer  Product  Safety  Commission.  574  F.  2d 
643  (1st  Cir.  1978). 


mine  if  OSHA's  statutory  language  requires  it  to  go 
beyond  the  data  it  has  used  in  the  past  and  follow  the 
guidelines  established  by  the  CPSC  rulings  for  the  use 
of  costs  v.  benefits  comparisons  in  its  decisionmaking 
procedures. 

The  available  evidence  strongly  indicates  that 
there  is  mounting  legislative  and  judicial  pressure  to 
formalize  the  use  of  CEA/CBA  in  many  decision- 
making areas  in  the  Federal  Government.  To  date, 
most  of  this  pressure  has  been  focused  on  the  regula- 
tory agencies.  It  is  uncertain  at  this  time  if  these  pres- 
sures will  expand  into  the  health  care  system. 
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In  addition  to  the  present  report,  several  back- 
ground papers  were  prepared  in  conjunction  with 
OTA's  assessment  The  Implications  of  Cost-Effec- 
tiveness Analysis  of  Medical  Technology,  and  these 
are  available  separately.  The  findings  and  options  of 
the  assessment  are  in  large  part  derived  from  the  in- 
formation generated  by  the  background  efforts. 

Background  Paper  #1:  Methodological  Issues  and 
Literature  Review,  includes  an  in-depth  examination 
of  the  decisionmaking  context  and  methodology  dis- 
cussions presented  in  this  report.  A  critique  of  the  lit- 
erature, a  bibliography  of  over  600  items,  and  ab- 
stracts of  over  70  studies  and  other  articles  are  also 
included. 

In  order  to  investigate  the  applicability  of  tech- 
niques to  assess  the  costs  and  benefits  of  medical 
technology,  19  case  studies  were  prepared.  All  19  are 
available  individually.  In  addition,  17  of  the  cases  are 
available  in  a  collected  volume,  Background  Paper 
#2:  Case  Studies  of  Medical  Technologies.  Some  of 
the  cases  represent  formal  cost-effectiveness  analysis 
(e.g.,  the  case  on  bone  marrow  transplants),  while 
others  represent  net  cost  or  "least  cost"  analysis  (e.g., 
the  case  on  certain  respiratory  therapies).  Other  cases 
illustrate  various  issues  such  as  the  difficulty  of  con- 
ducting cost-effectiveness  analysis  in  the  absence  of 
adequate  efficacy  and  safety  information  (e.g.,  the 
case  on  breast  cancer  surgery),  or  the  role  and  impact 
of  formal  analysis  on  policymaking  (e.g.,  the  case  on 
end-stage  renal  disease  interventions).  The  17  case 
studies  in  Background  Paper  #2  and  their  authors 
are: 

Artificial  Heart 

Deborah  P.  Lubeck 

John  P.  Bunker 
Automated  Multichannel  Chemistry  Analyzers 

Milton  C.  Weinstein 

Laurie  A.  Pearlman 
Bone  Marrow  Transplants 

Stuart  O.  Schweitzer 

C.  C.  Scalzi 
Breast  Cancer  Surgery 

Karen  Schachter 

Duncan  Neuhauser 
Cardiac  Radionuclide  Imaging 

William  B.  Stason 

Eric  Fortess 
Cervical  Cancer  Screening 

Bryan  R.  Luce 


Cimetidine  and  Peptic  Ulcer  Disease 

Harvey  V.  Fineberg 

Laurie  A.  Pearlman 
Colon  Cancer  Screening 

David  M.  Eddy 
CT  Scanning 

Judith  L.  Wagner 
Elective  Hysterectomy 

Carol  Korenbrot 

Ann  B.  Flood 

Michael  Higgins 

Noralou  Roos 

John  P.  Bunker 
End-Stage  Renal  Disease  Interventions 

Richard  A.  Rettig 
Gastrointestinal  Endoscopy 

Jonathan  A.  Showstack 

Steven  A.  Schroeder 
Neonatal  Intensive  Care 

Peter  Budetti 

Peggy  McManus 

Nancy  Barrand 

Lu  Ann  Heinen 
Nurse  Practitioners 

Lauren  LeRoy 

Sharon  Solkowitz 
Orthopedic  Joint  Prosthetic  Implants 

Judith  D.  Bentkover 

Philip  G.  Drew 
Periodontal  Disease  Interventions 

Richard  M.  Scheffler 

Sheldon  Rovin 
Selected  Respiratory  Therapies 

Richard  M.  Scheffler 

Morgan  Delaney 

The  18th  case  study  is  published  separately  as 
Background  Paper  #3:  The  Efficacy  and  Cost-Effec- 
tiveness of  Psychotherapy.  That  study  assesses  meth- 
odological and  substantive  issues  relating  to  the 
scope  of  psychotherapy,  the  evaluation  of  psycho- 
therapeutic efficacy,  and  the  applicability  of  cost-ef- 
fectiveness and  cost-benefit  analysis  in  assessing  psy- 
chotherapy. It  was  prepared  by  Leonard  Saxe  on  the 
basis  of  a  report  prepared  for  OTA  by  Brian  Yates 
and  Frederick  Newman.  The  19th  case  study  was  pre- 
pared for  OTA  by  Judith  Wagner  and  is  published 
separately  as  Background  Paper  #5:  Assessment  of 
Four  Common  X-Ray  Procedures. 

Background  Paper  #4:  The  Management  of  Health 
Care  Technology  in  Ten  Countries  is  an  analysis  of 
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the  policies,  programs,  and  methods,  including  cost- 
effectiveness  and  cost-benefit  techniques,  that  nine 
industrialized  nations  other  than  the  United  States 
use  to  manage  the  effects  of  medical  technology.  The 
experience  of  these  nine  countries  in  managing  medi- 
cal technology  is  compared  to  that  of  the  United 
States.  The  paper  on  the  United  States  and  the  com- 
parative analysis  were  prepared  by  OTA  staff,  as- 
sisted by  Louise  Russell.  The  authors  of  the  papers  on 
the  nine  foreign  countries  are: 
United  Kingdom 

Barbara  Stocking 
Canada 

Jack  Needleman 
Australia 

Sydney  Sax 
Japan 

Joel  Broida 
France 

Rebecca  Fuhrer 
Germany 

Karin  A.  Dumbaugh 


Netherlands 

L.  M.J.  Groot 
Iceland 

David  Gunnarson 

Duncan  vB.  Neuhauser 
Sweden 

Erik  H.  G.  Gaensler 

Egon  Jonsson 

Duncan  vB.  Neuhauser 

A  related  report  prepared  by  OTA  and  reviewed 
by  the  Advisory  Panel  to  the  overall  assessment  is  A 
Review  of  Selected  Federal  Vaccine  and  Immuniza- 
tion Policies.  That  study,  published  in  September  of 
1979,  examined  vaccine  research,  development,  and 
production;  vaccine  efficacy,  safety,  and  cost-effec- 
tiveness; liability  issues;  and  factors  affecting  the  use 
of  vaccines.  Pneumococcal  vaccine  was  used  as  a 
case  study,  and  a  cost-effectiveness  analysis  was  per- 
formed. 
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ACRONYMS 


ADAMHA  Alcohol,  Drug  Abuse,  and  Mental 

EPA 

Health  Administration  (PHS} 

EPSDT 

AHA 

American  Hospital  Association 

AIP 

annual  implementation  plan 

ESRD 

AMA 

American  Mpdical  Association 

FDA 

ANDA 

abbreviated  new  drus?  annliration 

FR 

AoA 

Administration  on  At?inp 

FY 

ASH 

Assistant  Secretarv  for  Health  (DHHS} 

■*  \.JJX  J  LCtl  I  i.  «_/ \_  V-  X  V_  L  CI  1  V    1  VJX    I  l\Tdl  Li  [    \  1  /  X  11  X<J  / 

GAO 

ASPE 

Assistant  Sprrptarv  for  Planninp  anrl 

*  ujio  mi  ii  i-/v_  v_  i  v.iai  y  i  vi  i.  lcaa  ix  ill  l  fci  cil  l  LI 

Evaluation  (DHHS) 

GHAA 

BC/BS 

Blue  Cross  and  Blue  Shield  Associations 

GHC 

BCHS 

Bureau  of  Communitv  Health  Sprvirps 

"M1  v •*        VI.    W11I111U1I1L  V    JL  1V.C1111  L   I  1  \_  v  V  lv.Cj 

(Health  Services  Administration) 

HCFA 

BHF 

Bureau  of  Health  Facilitips  (HR  A} 

!•> 'Ul  V.UU    \Al    1  1V.U1  U  I    X   C4  V-  X  X  X  L  1  \_  ,_>    \X                 I  / 

BHP 

Bureau  of  Health  Planning  (HRA) 

HEW 

BHPr 

Bureau  of  Health  Professions  (HRA) 

BMD 

Bureau  of  Medical  Devices  (FDA) 

HMO 

BMS 

Bureau  of  Medical  Services  (Health 

HRA 

Services  Administration} 

<-f  \-l    V  1  \_  V_  J  A    1  V4  1 1  111  11  J  LI  CI  11V/1  1  / 

HSA 

BOB 

Burpau  of  Biologies  (FDA} 

l-'w.l\-C4VM    vl    1/lVlv/tlV.ij    y  1    1  /  A    \.  1 

HSI 

BOD 

Bureau  of  Drugs  (FDA) 

HSP 

BOE 

Bureau  of  Epidemiology  (CDC) 

HSQB 

BOL 

Bureau  of  Laboratories  (CDC) 

BRH 

Bureau  of  Radiological  Health  (FDA) 

IDE 

CBA 

cost-benefit  analysis 

IHS 

CBO 

Congressional  Budget  Office  (U.S. 

Congress) 

IND 

CDC 

Center  for  Disease  Control  (PHS) 

CEA 

cost-effectiveness  analysis 

IOM 

CEA/CBA  cost-effectiveness  analysis/cost-benefit 

IPA 

analysis  (when  referred  to  as  a  class  of 

KFMCP 

analytical  techniques) 

MCE 

CFR 

Code  of  Federal  Regulations 

CHP 

Comprehensive  Health  Planning 

NAS 

CMHC 

community  mental  health  center 

NASA 

CON 

certificate  of  need 

CPSC 

Consumer  Product  Safety  Commission 

NCHCT 

CT 

computed  tomography  (scanner) 

DARP 

Drug  Abuse  Reporting  Program  (NIDA) 

NCHS 

DES 

diethylstilbestrol 

DESI 

Drug  Efficacy  Study  Implementation 

NCHSR 

1  N  V_l  1l/1\ 

(project) 

DHEW 

Department  of  Health,  Education,  and 

NCI 

Welfare  (now  DHHS) 

NDA 

DHHS 

Department  of  Health  and  Human 

NEI 

Services  (formerly  DHEW) 

NEPA 

DOD 

Department  of  Defense 

NHLBI 

DOE 

Department  of  Energy 

DOL 

Deparment  of  Labor 

NIA 

DRG 

Division  of  Research  Grants  (NIH) 

NIAAA 

EFM 

electronic  fetal  monitoring  - 

Environmental  Protection  Agency 
Early  and  Periodic  Screening,  Diagnosis, 

and  Treatment  (Program)  (HEW) 
end-stage  renal  disease 
Food  and  Drug  Administration  (PHS) 
Federal  Register 
fiscal  year 

General  Accounting  Office  (U.S. 

Congress) 
Group  Health  Association  of  America 
Group  Health  Cooperative;  Seattle, 

Wash. 

Health  Care  Financing  Administration 
(DHHS) 

Department  of  Health,  Education,  and 

Welfare  (now  DHHS) 
health  maintenance  organization 
Health  Resources  Administration  (PHS) 
health  systems  agency 
health  status  index 
health  systems  plan 
Health  Standards  and  Quality  Bureau 

(HCFA) 

investigational  device  exemption 
Indian  Health  Service  (Health  Services 

Administration) 
notice  of  claimed  investigational 

exemption  for  a  new  drug 
Institute  of  Medicine  (NAS) 
individual  practice  association  (HMO) 
Kaiser  Foundation  Medical  Care  Plan 
medical  care  evaluation  studies  (by 

PSROs) 
National  Academy  of  Sciences 
National  Aeronautics  and  Space 

Administration 
National  Center  for  Health  Care 

Technology  (OASH) 
National  Center  for  Health  Statistics 

(OASH) 

National  Center  for  Health  Services 

Research  (OASH) 
National  Cancer  Institute  (NIH) 
new  drug  application 
National  Eye  Institute  (NIH) 
National  Environmental  Policy  Act 
National  Heart,  Lung,  and  Blood 

Institute  (NIH) 
National  Institute  on  Aging  (NIH) 
National  Institute  on  Alcohol  Abuse  and 

Alcoholism  (ADAMHA) 
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NIAID 

National  Institute  of  Allergy  and 

OMAR 

Office  of  Medical  Applications  of 

Infectious  Diseases  (NIH) 

Research  (NIH) 

NIAMDD 

National  Institute  of  Arthritis, 

OPE 

Office  of  Planning  and  Evaluation 

Metabolism,  and  Digestive  Diseases 

(OASH) 

(NIH) 

OPEL 

Office  of  Planning,  Evaluation,  and 

NICHD 

National  Institute  of  Child  Health  and 

Legislation 

Human  Development  (NIH) 

OPPR 

Office  of  Policy  Planning  and  Research 

NIDA 

National  Institute  on  Drug  Abuse 

(HCFA) 

(ADAMHA) 

OSHA 

Occupational  Safety  and  Health 

NIDR 

National  Institute  of  Dental  Research 

Administration  (DOL) 

(NIH) 

OTA 

Office  of  Technology  Assessment  (U.S. 

NIEHS 

National  Institute  of  Enviromental 

Congress) 

Health  Sciences  (NIH) 

OTC 

over-the-counter  (Drug  Review 

NIGMS 

National  Institute  of  General  Medical 

Program)  (FDA) 

Sciences  (NIH) 

PDP 

product  development  protocol 

NIH 

National  Institutes  of  Health  (PHS) 

PGP 

prepaid  group  practice  (HMO) 

NIMH 

National  Institute  of  Mental  Health 

PHS 

Public  Health  Service  (DHHS) 

(ADAMHA) 

PPB 

planning-programing-budgeting 

NINCDS 

National  Institute  of  Neurological  and 

PMA 

Pharmaceutical  Manufacturers' 

Communicative  Disorders  and  Stroke 

Association 

(NIH) 

PSRO 

Professional  Standards  Review 

NIOSH 

National  Institute  for  Occupational 

Organization,  or  Office  of  PSRO 

Safety  and  Health  (CDC) 

(HCFA) 

NLM 

National  Library  of  Medicine  (NIH) 

QALY 

quality-adjusted  life  year 

NSF 

National  Science  Foundation 

RFP 

request  for  proposal 

OASH 

Office  of  the  Assistant  Secretary  for 

RMP 

Regional  Medical  Program 

Health  (DHHS) 

SHCC 

State  Health  Coordinating  Council 

OASPE 

Office  of  the  Assistant  Secretary  for 

SHPDA 

State  health  planning  and  development 

Planning  and  Education  (DHHS) 

agency 

ODPHP 

Office  of  Disease  Prevention  and  Health 

SSA 

Social  Security  Administration  (DHHSj 

Promotion  (OASH) 

TOPS 

Treatment  Outcome  Prospective  Study 

OHDS 

Office  of  Human  Development  Services 

(NIDA) 

(DHHS) 

VA 

Veterans  Administration 

OHMO 

Office  of  Health  Maintenance 

ZBB 

zero  based  budgeting 

Organizations  (OASH) 

OHRST 

Office  of  Health  Research,  Statistics,  and 

Technology  (OASH) 

GLOSSARY 

Ambulatory  medical  care:  Medical  goods  and  serv- 
ices rendered  to  a  patient  not  admitted  to  a  hospi- 
tal or  other  inpatient  health  care  facility,  including 
such  items  as  physician  office  visits,  outpatient 
laboratory  diagnostic  services,  and  outpatient  pre- 
scription drugs. 

Biomedical  and  behavioral  research:  A  combination 
of  biological,  medical,  psychological,  social,  and 
physical  scientific  investigations  focused  on  eradi- 
cating disease  and  generating  new  scientific  knowl- 
edge. 

Capitation  financing  method:  The  method  of  paying 
for  medical  care  on  a  fixed,  periodic  prepayment 


basis  per  individual  enrolled  in  a  health  plan.  Pay- 
ment by  "capitation"  implies  that  the  amount  paid 
by  the  individual  is  independent  of  the  number  of 
services  that  individual  has  received. 
CEA/CBA:  A  composite  term  referring  to  a  family  of 
analytical  techniques  that  are  employed  to  com- 
pare costs  and  benefits  of  programs  or  technol- 
ogies. Literally,  the  term  as  used  in  this  assessment 
means  "cost-effectiveness  analysis /cost-benefit 
analysis." 

Certificate  of  need  (CON):  A  regulatory  planning 
mechanism  required  by  the  National  Health  Plan- 
ning and  Resources  Development  Act  to  control 
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large  health  care  capital  expenditures.  CON  appli- 
cations by  institutions  are  reviewed  by  local  health 
systems  agencies,  who  recommend  approval  or 
disapproval;  they  are  denied  or  approved  by  State 
health  planning  agencies  (SHPDAs). 

Consensus  development  conference:  A  process  in 
which  knowledgeable  biomedical  researchers, 
practicing  health  professionals,  and  others,  as  ap- 
propriate, are  brought  together  by  NIH  to  explore 
publicly  the  scientific  background,  state  of  knowl- 
edge, proper  use(s),  and  any  other  issues  pertinent 
to  the  technology  under  consideration. 

Controlled  clinical  trial:  An  experimental  design  by 
which  human  or  animal  subjects  are  assigned,  in 
accordance  with  predetermined  rules,  either  to  an 
experimental  group  (in  which  subjects  receive  a 
clinical  intervention  or  dosage  level  of  uncertain 
efficacy  or  safety)  or  to  a  control  group  (in  which 
subjects  receive  some  other  intervention  or  dosage 
level,  usually  the  customary  or  conventional  one, 
or  a  placebo).  If  the  predetermined  rules  specify 
that  the  subjects  are  assigned  to  groups  randomly, 
the  result  is  a  randomized  controlled  clinical  trial. 
The  vast  majority  of  randomized  clinical  trials  are 
also  controlled  trials. 

Cost-benefit  analysis:  An  analytical  technique  that 
compares  the  costs  of  a  project  or  technological  ap- 
plication to  the  resultant  benefits,  with  both  costs 
and  benefits  expressed  by  the  same  measure.  This 
measure  is  nearly  always  monetary. 

Cost-effectiveness  analysis:  An  analytical  technique 
that  compares  the  costs  of  a  project  or  of  alter- 
native projects  to  the  resultant  benefits,  with  cost 
and  benefits/effectiveness  not  expressed  by  the 
same  measure.  Costs  are  usually  expressed  in  dol- 
lars, but  benefits/effectiveness  are  ordinarily  ex- 
pressed in  terms  such  as  "lives  saved,"  "disability 
avoided,"  "quality-adjusted  life  years  (QALYs) 
saved,"  or  any  other  relevant  objectives.  Also, 
when  benefits /effectiveness  are  difficult  to  express 
in  a  common  metric,  they  may  be  presented  as  an 
"array." 

Device  (medical):  Any  physical  item,  excluding 
drugs,  used  in  medical  care  (including  instruments, 
apparatus,  machines,  implants,  and  reagents). 

Discount  rate:  A  factor  used  in  economic  analysis  to 
reduce  to  present  value  those  costs  and  effects  that 
occur  in  future  years.  Discounting  is  based  on  two 
premises:  1)  individuals  prefer  to  receive  benefits 
today  rather  than  in  the  future,  and  2)  resources  in- 
vested today  in  alternative  programs  could  earn  a 
return  over  time. 

Drug:  Any  chemical  or  biological  substance  that  may 
be  applied  to,  ingested  by,  or  injected  into  humans 


in  order  to  prevent,  treat,  or  diagnose  disease  or 
other  medical  conditions. 

Effectiveness:  Same  as  efficacy  (see  below)  except 
that  it  refers  to  ".  .  .  average  conditions  of  use." 

Efficacy:  The  probability  of  benefit  to  individuals  in 
a  defined  population  from  a  medical  technology 
applied  for  a  given  medical  problem  under  ideal 
conditions  of  use. 

Fee-for-service:  A  method  of  paying  for  medical  care 
on  a  retrospective  basis  by  which  each  service  ac- 
tually received  by  an  individual  bears  a  related 
charge. 

Health  maintenance  organization:  A  health  care 
organization  that  acts  as  both  insuror  and  provider 
of  comprehensive  medical  services  by  a  defined  set 
of  physicians  to  a  voluntarily  enrolled  population 
paying  a  prospective  per  capita  fee  (i.e.,  paying  by 
"capitation"). 

Health  status  index  (HSI):  A  composite  measure  of 
well-being  of  an  individual  or  a  population.  An 
HSI  may  include  any  two  or  more  measures  such 
as  morbidity,  mortality,  level  of  function  or  mobil- 
ity, and  potential  for  developing  disease. 

Health  services  research:  A  field  of  inquiry  that 
focuses  on  the  structure,  production,  distribution, 
and  effects  of  delivering  personal  health  services. 

Health  systems  agency  (HSA):  One  of  the  local 
health  planning  agencies  which  are  charged  with 
developing  local  health  planning  goals  and  imple- 
menting plans  in  consonance  with  State  and  Na- 
tional health  care  goals.  HSAs  are  federally  funded 
and  are  governed  by  a  body  which  is  broadly  rep- 
resentative of  both  provider  and  consumer  inter- 
ests, the  latter  being  in  the  majority.  The  country  is 
divided  into  areas  each  of  which  is  served  by  an 
HSA. 

Human  capital:  An  economic  concept  used  to  assess 
"livelihood,"  or  the  earnings  potential  of  an  in- 
dividual. It  has  often  been  used  as  a  proxy  for  the 
value  of  life  in  terms  of  an  individual's  productive 
capacity  to  society. 

Investigational  new  drug  (IND)  application:  Short 
for  "notice  of  claimed  investigational  exemption 
for  a  new  drug."  An  IND  application  is  submitted 
to  FDA  by  a  drug's  sponsor.  It  is  a  request  for  per- 
mission to  use  the  drug  in  humans  in  order  to  in- 
vestigate the  drug's  safety  and  efficacy  for  specific 
conditions.  It  must  include  the  results  of  toxicity 
studies  in  animals,  the  qualifications  of  the  investi- 
gators, and  the  design  of  the  proposed  clinical 
studies. 

Marginal  benefit:  An  economic  concept  referring  to 
the  additional  benefit  achieved  by  incurring  an  ad- 
ditional unit  of  cost. 
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Marginal  cost:  An  economic  concept  referring  to  the 
additional  cost  of  achieving  one  more  unit  of  bene- 
fit. 

Medical  care  evaluation  studies:  Intensive  retrospec- 
tive reviews  of  patients'  records  to  evaluate  the 
quality,  variation,  and  utilization  of  health  care 
services  delivered  to  the  patient. 

Medical  technology:  The  drugs,  devices,  and  medical 
and  surgical  procedures  used  in  medical  care,  and 
the  organizational  and  support  systems  within 
which  such  care  is  provided. 

Morbidity:  Illness,  injury,  impairment,  or  disability 
in  an  individual. 

Mortality:  The  death  of  an  individual;  often  used  in 
epidemiological  studies  where  mortality  rates  for  a 
population  for  a  certain  disease  or  injury  are  calcu- 
lated. 

Net  cost  analysis:  A  form  of  CEA/CBA  (see  above) 
that  concentrates  on  costs,  with  less  attention  paid 
to  analyzing  outcomes  in  terms  of  health  benefit. 
When  alternatives  are  under  study,  their  efficacy  is 
often  assumed  to  be  equal. 

New  drug  application  (NDA):  An  application  to 
FDA  by  the  sponsor  of  a  new  drug  for  permission 
to  market  the  drug.  The  NDA  must  provide  infor- 
mation that  demonstrates  the  safety  and  efficacy  of 
the  drug. 

Procedure:  A  medical  technology  involving  any 
combination  of  drugs,  devices,  and  provider  skills 
and  abilities.  Appendectomy,  for  example,  may  in- 
volve at  least  drugs  (for  anesthesia),  monitoring 
devices,  surgical  devices,  and  physicians',  nurses', 
and  support  staffs'  skilled  actions. 

Professional  Standards  Review  Organization:  Com- 
munity-based nonprofit  agencies  directed  by 
physicians  that  monitor  the  quality  and  appropri- 
ateness of  institutional  health  care  provided  to 
medicare  and  medicaid  beneficiaries. 

Quality-adjusted  life  year  (QALY):  A  health  status 
index  in  which  1  year  of  life  is  adjusted  for  various 
types  and  degrees  of  disability  to  yield  1  year  of 
healthy  life.  QALYs  are  sometimes  used  to  meas- 


ure in  common  terms  the  effects  on  morbidity  and 
mortality  of  health  care  technologies  or  programs. 

Risk:  A  measure  of  the  probability  of  an  adverse  or 
untoward  outcome's  occurring  and  the  severity  of 
the  resultant  harm  to  health  of  individuals  in  a  de- 
fined population  associated  with  use  of  a  medical 
technology  applied  for  a  given  medical  problem 
under  specified  conditions  of  use. 

Risk-benefit  analysis:  The  formal  comparison  of  the 
probability  and  level  of  adverse  or  untoward  out- 
comes versus  positive  outcomes  for  any  given  ac- 
tion. The  comparison  of  outcomes  does  not  take 
into  consideration  the  resource  costs  involved  in 
the  intended  action. 

Safety:  A  judgment  of  the  acceptability  of  risk  (see 
above)  in  a  specified  situation. 

State  Health  Coordinating  Council  (SHCC):  The 
State  advisory  body  to  the  State  health  planning 
and  development  agency  (SHPDA).  SHCCs  con- 
sist of  a  consumer  majority,  mostly  from  health 
system  agencies'  representatives  who  are  ap- 
pointed by  the  Governor,  and  are  responsible  for 
approving  the  State  health  plan  which  is  submitted 
by  the  SHPDA. 

State  health  planning  and  development  agency 
(SHPDA):  State-level  health  planning  organiza- 
tions which  are  federally  mandated  to  develop  and 
implement  statewide  health  plans  in  consonance 
with  national  goals  and  local  planning  objectives. 
SHPDAs  must  approve  or  disapprove  health  sys- 
tem agencies'  recommendations  regarding  certifi- 
cate of  need  for  large  new  capital  investments. 

Willingness-to-pay:  An  economic  concept  used  to  as- 
sess the  monetary  value  of  life  in  terms  of  what  an 
individual  is  "willing  to  pay"  to  prolong  life  or 
postpone  death.  The  willingness-to-pay  technique 
is  used  to  try  to  assess  how  much  an  individual  val- 
ues his  or  her  own  life.  Sometimes,  the  technique  is 
also  used  to  assess  how  much  an  individual  is  will- 
ing to  pay  to  decrease  risk  to  others  or  to  prolong 
others'  lives. 
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